devopsy.kinsta.cloud

Tag: VM

  • Our Infrastructure is Still Expanding

    Our Infrastructure is Still Expanding

    You would think that sooner or later, marketers and early adopters would take a rest, but it seems that if one group slows down for a bit, another picks up. Our infrastructure is expanding in almost every possible way, and this creates more of a burden on every aspect of DevOps, specifically in IT.

    We have more security needs because we have a larger footprint, for example. We have long had APIs, but now we have massive API groups that are used and re-used, and while standard code scanning works with the source, the companies doing “API security” start with security of the interface, not the code. That is slowly changing, but API management is also impacted, and organizations that determined they didn’t need API management because adjacent markets were serving enough of their needs are slowly discovering that with proliferation comes the need to look at API management again.

    I stay up to date on most new technology because, as an industry analyst, what is coming can inform my discussions of what a given technology space looks like today. But even for me—someone who is not spending a huge chunk of my day coding or working on the toolchain or perfecting the new deployment platform—it can be hard to stay abreast of everything. While one of the foundational tenets of DevOps was no specialization and a shared workload, I think everyone now agrees with what a few of us were saying from day one: Some specialized skills are still needed, except in corner cases. But we certainly are sharing workloads more broadly and far more responsive, so DevOps is absolutely a massive success. Site reliability engineering (SRE) extends that success but focuses more on Ops. I’m really not a huge fan. I like that DevOps wanted to merge the processes and make the management of applications more holistic; I don’t think re-dividing it is the best idea out there. Like every title, “SRE” is interpreted pretty broadly. Some roles really involve DevOps (but SRE sounds cooler), and other organizations are just replacing ‘developer’ with DevOps and ‘operations’ with SRE. However, most are using SRE to manage the entire environment in which the app runs.

    Where are we going? Constant change. That’s the answer. I’ve waited for the breather for a couple of decades, but since VMs were popularized, it has been non-stop and will continue to be.

    This means a lot to you, to your viability as an employee, and to hiring. With the glut of people on the market, we’re seeing a resurgence of “must know every tool in our toolchain” type position postings. Of course, you must not, in a world where the combination of languages, frameworks, CI/CD tools, alerting, and monitoring tools would make a great college-level combinatorics equation. You add value by being adaptable. Both to your current employer and to future ones. The key thing you should seek to demonstrate is your ability to pick up and master new technologies when required. Do it on your current job to show worth, and keep a record of it for your next job. I’ve been a hiring manager off and on forever, and when the market is flooded with good candidates, that is the type of differentiator that gets the team’s attention. Do it; make it something worth doing and document that you did it. You won’t regret it.

    A group of us were helping a recent grad figure out what to do with their career a couple weeks ago, and the consensus was “avoid AI until it shakes out.” This is not the first time I’ve heard that advice—it’s not even the first group I was a part of giving that advice. And I agree in the general sense, but certainly in places where it is being implemented by all vendors in the space and it adds value, you do need to be aware of how it works and what you can do with it, and a project is one way to get that knowledge.

    It is still a rolling opportunity, and you are still keeping the lights on. I’ve seen people who worry they’re not doing enough. Any employer worth working for wants you to keep the systems running first, and everything else after—so if you’re doing that, you’re doing enough. Now go beyond. And keep rocking it. This new technology hasn’t implemented itself, and it isn’t self-maintaining (yet) … That’s all you.

  • Convert COBOL Easily? Yes, Please!

    Convert COBOL Easily? Yes, Please!

    I was reading Mike Vizard’s article about IBM previewing an AI tool to convert COBOL to Java and mentally cheered this move. No, seriously—massively cheered this move.

    There are a lot of uses of large language models (LLM) right now, and many of them are not all that exciting, in my opinion. This one, though? This is huge. In the course of my career, I have been involved in three projects to convert COBOL and one to convert FORTRAN, with some projects that required interfacing with RPG III systems. In all of these cases, the costs were prohibitive. Two of the three COBOL projects never made it past estimates because the cost of resources who know both COBOL and Java were prohibitive, as is the complexity of most mainframe applications as they were modified over the years. The Fortran project never got past resourcing, and the RPG III interfaces were just that. We had to pull the data as it was presented by RPG III systems and only act on our copy, with a separate process used to update the core systems. Why? Because modifying the RPG was impractical, but we could be agile. It was … painful, to put it mildly.

    What all of these projects shared was the lack of a skilled workforce. And what IBM is offering is an LLM that acts as that workforce. If you get the application into Java, there is a large pool of developers that can then troubleshoot it. While the environment issues will still exist and people skilled in tools like JCL are increasingly hard to find, that part can be rebuilt in the new environment. (One question I have about IBM’s project is, “Can it convert to Java for a Linux LPAR/VM?” I suspect that this iteration converts within z/OS, but it would be cool to see increased options in the future.)

    Honestly, this trend will only increase. While IBM’s use of LLM is aimed at moving COBOL to Java, there are an incredible number of lines of Java out there to train on, so soon enough, there will be other conversions in and out of Java. And that’s all good news. We recreate the wheel a lot. The longer-term implication of AI-assisted conversion is that we won’t have to. Think about it: We could feed in a mathematical routine in Fortran and get the language of our choice out the other side.

    Way back in the day, a collection of compilers named TopSpeed let you develop in any of several languages, even mixing and matching to create applications that used each language (and developer) for their strengths. This is the TopSpeed dream writ large. (Disclaimer: I was a huge TopSpeed fan. Bought their intro kit and wrote systems mixing C/Pascal/ASM without having to mess with things like parameter sequences. Yeah, I was cool.)

    We have had too few developers for a very long time. Now, we are about to have a world where developers can focus on truly unique functionality, and the rest is generated based on the mass of implementations we already have. Think in terms of sorting algorithms—99% of the sorting problems out there were mastered decades ago, and yet we frequently see people re-implementing, even today. Now we can tell a system, “Generate a tree sort,” and just make calls. Every time we implement a new language, this type of thing has to be re-implemented, and often it is re-implemented because we don’t know about the available solutions. In the future, we will have solutions at our fingertips.

    And as with the other benefits of automation, we’ll do less busy work and add more business value. While there will be some pain, I think in the long term that is a huge benefit. And you’ll still be there, making things go zoom.

  • Forget Change, Embrace Stability

    Forget Change, Embrace Stability

    DevOps is about change. It was introduced to accelerate change, and it has opened up a whole new world of tools and possibilities that are all based on the premise of changing IT. Starting with Agile, we now have DevOps variants and subprocesses that run the entire software development life cycle (SDLC) process.

    Want to make more frequent updates that can regularly show demonstrable progress to business owners? Agile and CI! Need to get the app settled in its inevitable home with automation? You need CD! Want testing and zero-day protections? Start with DevSecOps built into the CI/CD process and add in application and API security! Want to fully automate deployment in the whay builds are automated? Grab a GitOps tool and let’s goooo! The list goes on and on. Literally. We’ve almost got more tools than we have names for spaces at this point.

    And all of those tools introduce change. Most of them introduce a massive amount of change. You don’t get to GitOps without breaking some of your existing processes and reworking them. We have fallen in love with change in IT.

    But there is a funny thing that’s happened along the way; I’ve mentioned it before, but never from this perspective. All great changes require time for consolidation. Be it stock market swings or revolutions, afterward, there is a period of consolidation. In IT, containers are a good example. When they became capable of actually running workloads, IT experienced a period of time when making all the various machines run in VMs was the thing. People weren’t (generally) looking for the new big change in the process, they were just glad that those one-off machines that had custom hardware could now be moved to something more manageable and that spinning up servers could be done in days or weeks instead of the months it used to take when you had to order hardware. But that consolidation created organizational standards and stability that the organization enjoyed for years – even after containers rose to challenge VMs and some organizations even today.

    So every once in a while, take a breather, consolidate gains, create standards that have meaning for your organization, then continue on with whatever the shiny new change is. Make sure that once you have identified, for example, the need for code scanners that can work within or that are complementary to the integrated development environment (IDE), that you have one scanner—or one scanner per programming language—to maintain and monitor instead of one tool per project. That saves time and creates stability, both of which will serve the organization moving forward.

    Change is here to stay; it’s the only constant. IT is caught in a cycle of change followed by change, with the next wave driven by generative AI. But in times of change, seek stability and, in times of stability, seek changes that can have a positive impact. So go standardize and organize before the next wave hits.

    Mostly, the changes will serve the organization by freeing up your time and letting you spend more of your time rocking the IT world. Those systems stay up because of you. More time is more innovation, and more stability means less burnout. All of which is good for the organization.

  • NGINX Expands Portfolio to Include Service Mesh

    NGINX Expands Portfolio to Include Service Mesh

    NGINX at its online Sprint conference today launched a bevy of offerings including a service mesh as part of an expanded effort to meld network and security operations within the context of a larger DevOps workflow.

    Rob Whiteley, chief marketing officer for the subsidiary of F5 Networks, said NGINX Service Mesh 1.0 provides an easier alternative to deploy a service mesh based on the open source NGINX proxy server software already widely employed in enterprise IT environments. That lighter-weight service mesh is also integrated with the NGNIX ingress controller to enable it to be employed across microservices spanning both legacy virtual machine environments and Kubernetes clusters, he said.

    In general, Whiteley noted, most organizations will not need a service mesh unless they are looking to integrate more than a thousand microservices. If an organization only has hundreds of microservices, an ingress controller should suffice, and anything involving less than a hundred microservices can be handled by proxy software.

    NGINX Service Mesh 1.0 will be offered by NGINIX and F5 Networks will continue to offer a service mesh based on the Istio project for Kubernetes environments that relies on Envoy proxy software, which is not as lightweight as the NGINX platform, noted Whiteley. DevOps teams will have to decide which service mesh to deploy.

    In the meantime, NGINX is also updating its NGINX Controller to add support richer analytics, a declarative application programming interface (API) for continuous integration/continuous delivery (CI/CD) integration, role-based access controls and enhancements to the developer portal for API management.

    NGINX is also driving tighter integration between Controller and NGINX App Protect, a web application firewall that will inherit the API, analytics and self-service capabilities of Controller. NGINIX App Protect is also now being integrated with NGINX Ingress Controller for Kubernetes. The Controller App Security tool that enables those integrations is in private beta now, with general availability expected by end of 2020.

    Finally, NGINX is now leveraging services made available by F5 Networks via software-as-a-service (SaaS) platforms in the form of NGINX Analytics Cloud, which replaces NGINX Amplify. That capability will be available in the first quarter of 2021.

    In general, Whiteley said NGINX is trying to serve the needs of traditional network operation teams that want to provide self-service capabilities to development teams as well as via programmatic interfaces to DevOps teams. In many cases, enterprise IT organizations are employing both approaches to expose network services, noted Whiteley.

    Of course, the issue that ultimately might drive the convergence of NetOps and DevOps is the rise of the service mesh. In addition to integrating microservices, the service mesh provides developers with a consistent set of interfaces for invoking network and security services spanning a dizzying array of servers and appliances. As that transition occurs, IT teams will become much more agile than they are today. The challenge, of course, will be melding network operations and DevOps cultures that today don’t have all that much in common.

  • Docker vs VMs

    Docker vs VMs

    All applications have their own dependencies, which include both software and hardware resources. Docker containers bring numerous unknown tags to the chart as compared to existing technologies in use. In terms of technology, it’s quite interesting in scenarios where it helps in promoting cloud portability by running the same applications in different virtual environments. Docker is an open platform for developers, it’s a mechanism that helps in isolating the dependencies per each application by packing them into containers. Containers are scalable and safer to use and deploy as compared to previous approaches.

    Virtual machines are used extensively in cloud computing. Isolation and resource control have continually been achieved through the use of virtual machines. Virtual machine loads a full OS with its own memory management and enable applications to be more efficient and secure while ensuring their high availability.

    How are Docker Containers different from a Virtual Machine?

    Virtual machines have a full OS with its own memory management installed with the associated overhead of virtual device drivers. In a virtual machine, valuable resources are emulated for the guest OS and hypervisor, which makes it possible to run many instances of one or more operating systems in parallel on a single machine (or host). Every guest OS runs as an individual entity from the host system.

    On the other hand Docker containers are executed with the Docker engine rather than the hypervisor. Containers are therefore smaller than Virtual Machines and enable faster start up with better performance, less isolation and greater compatibility possible due to sharing of the host’s kernel.

    Docker Containers versus Virtual Machines:

    When it comes to comparing the two, it could be said that Docker Containers have much more potential than Virtual Machines. It’s evident as Docker Containers are able to share a single kernel and share application libraries. Containers present a lower system overhead than Virtual Machines and performance of the application inside a container is generally same or better as compared to the same application running within a Virtual Machine.

    There is one key metric where Docker Containers are weaker than Virtual Machines, and that’s “Isolation”. Intel’s VT-d and VT- x technologies have provided Virtual Machines with ring-1 hardware isolation of which, it takes full advantage. It helps Virtual Machines from breaking down and interfering with each other. Docker Containers yet don’t have any hardware isolation, thus making them receptive to exploits.

    As compared to virtual machines, containers can be faster and less resource heavy as long as the user is willing to stick to a single platform to provide the shared OS. A virtual machine could take up several minutes to create and launch whereas a container can be created and launched just in a few seconds. Applications contained in containers offer superior performance, compared to running the application within a virtual machine.

    There is an estimation being done by Docker that application running in a container can go twice as fast as one in a virtual machine. Also, a single server can pack more than one containers as OS is not duplicated for each application.

    Virtual Machines and Containers: better together

    You can sometimes use a hybrid approach which uses both VM and Docker. There are also workloads which are best suited for physical hardware. If both are placed in a hybrid approach, it might lead to a better and efficient scenario. With this Hybrid setup, users can benefit from the advantages if they have workloads that fit the model. 

    Following are a few of them, that explain how they work together as a Hybrid:

    1). Docker Containers and Virtual Machines by themselves are not sufficient to operate an application in production. So one should be considering how are the Docker Containers going to run in an enterprise data center.

    2). Application probability and enabling the accordant provisioning of the application across infrastructure is provided by containers. But other operational requirements such as security, performance and capacity management and various management tool integrations are still a challenge in front of Docker Containers, thus leaving everyone in a big puzzle.

    3). Security isolation can be equally achieved by both Docker Containers and Virtual Machines.

    4). Docker Containers can run inside Virtual Machines though they are positioned as two separate technologies and provide them with pros like proven isolation, security properties, mobility, dynamic virtual networking, software-defined storage and massive ecosystem.

    Apples to apples comparison: On a physical host with a certain configuration and Virtual Machines with the same configuration running an identical running same number of docker Containers with the same performance on both?

    Who wins amongst the two?

    Answer to this question so far cannot be ascertained but depending upon their configurations and constraints one could say that containers are overcoming virtual machines. Application design is the one standpoint suggesting which one of the two should be chosen. If application is designed to provide scalability and high availability then containers are the best choice else application can be placed in a virtual machine, though Docker containers have surely challenged  virtualization market with containers. Well, keeping the debate aside, it is easy to say that containers in Virtual Machines are twice as robust as one without the other.

    Conclusion:-

    Docker Containers are transforming the DevOps (developer operations) landscape as an important tool in DevOps arsenal. The use cases for Docker Containers within the realm of DevOps are plenty. Running apps over Docker Containers and then deploying anywhere (Cloud or on-premise or any flavor of linux) is a reality now.

    Working in heterogeneous environment, Virtual Machines provide high flexibility whereas Docker containers’ prime focus is on applications and their dependencies. The prime goal of the article is to compare the both on the basis of performance. Docker Containers promise to port application stacks across cloud easily by using each cloud’s Virtual Machine environment to deal with the cloud. This represents a useful capability that has to be implemented in a more complex and tedious way today without Docker Containers. Its not about abandoning Virtual Machines but to advocate in favor and design for the use of Docker Containers in addition to Virtual Machines when necessary. We don’t think Docker Containers could completely wipe out Virtual Machines.