devopsy.kinsta.cloud

Author: Don Macvittie

  • New Survey Highlights Codebase Size Problems

    New Survey Highlights Codebase Size Problems

    Sourcegraph and Dimensional Research have released a survey of developers at large organizations, which shows a massive growth in codebase size, number of repositories in use and complexity concerns.

    “The Emergence of Big Code – A 2020 Survey of Software Professionals” is definitely intriguing reading, and a few of the results are eye-opening—such as the difference between what development managers think developers are using to search code and what developers say they are actually using. But most of it simply validates what we all knew: Codebases are growing at an alarming rate, through a variety of growth vectors—support for new platforms, use of open source and internal development. Adding environment to code à la GitOps, systems reliability engineering and/or environment automation is adding a layer of completely new source and a layer of complexity not normally in the developer’s realm of immediate concern.

    I’ve been working with Accelerated Strategies Group over the last few months, and we’re looking at tools in this market. The ability to quickly get up to speed on large codebases and the ability to find a specific area in a codebase that is massive and/or spans repositories is a very real need that is not currently well-served. This piques my interest in what Sourcegraph brings to the table, but at the moment we’re focused on the survey, so perhaps in a future blog we’ll talk about Sourcegraph.

    While there is a lot packed into the survey, I think my favorite part is crammed in with other “What would you find useful?” results (the slide is below). Fifty-eight percent of respondents—all of whom are developers—chose, “Finding code by querying semantic relationships, not just matching a string.” As developers, these respondents no doubt are aware of the complexities of this type of feature, particularly in environments with a variety of programming languages and scripts (be they bash, SQL, whatever) all working together. This is huge, and first iterations would be shallow, but is a worthy goal. I’m intrigued to see how well SourceGraph et al solve it.

    Source: The Emergence of Big Code – A 2020 Survey of Software Professionals

    It is also intriguing to see more than half of respondents want the ability to explore ” … both known and unknown code with code intelligence that provides contextual understanding.” That, too, is a tall order. Code analysis has gotten better over time, but contextual analysis of large codebases is notoriously difficult. I’ll be intrigued in the near future to play with this concept and tools that offer it to see how reliable they are and how much human guidance is accepted and/or required to make it a reality.

    Code change velocity reporting and contextual information about code are both available and pretty solid from the majority of full-stack DevOps vendors. Perhaps that is why that particular response only netted 45% and 43% of responses, respectively—the other 50+% are already using them.

    We’re in an interesting time when consolidation appears to be on the upswing and the needs of developers and DevOps teams working in large codebases are becoming more clear and more complex at the same time. Over the next few years the needs reported in this survey will likely be replaced by more specific needs as these are fulfilled and find general use. The question of databases and database integration in large codebase migrations, for example, has these needs, but has other needs closer to the problem domain. As these general issues are resolved, expect your vendors to turn to those more specific, but every bit as complex, problems.

    And just keep rocking it. Find out what the tools and companies already in use at your org have added, and if those capabilities fit with your needs or can enhance your Dev/DevOps exercises. Keep the lights on, keep customers happy and watch for tools to help you make it even better.

  • Linux Foundation Adds Entry-Level Certification

    Linux Foundation Adds Entry-Level Certification

    The Linux Foundation has announced the development of a new entry-level certification exam to complement their existing Linux Foundation Certified Sysadmin (LFCS) and Linux Foundation Certified Engineer (LFCE) exams. This new certification, the Linux Foundation Certified IT Associate (LFCA), targets people just moving into systems administration.

    The certification will focus on the test taker’s knowledge of the fundamentals of system administration. We are in a complex world with a lot of systems administration options, and the LFCA intends to touch a little bit on each of the areas a new admin might run into. The test is still in development, but the areas that The Linux Foundation intends to focus on include:

    • Linux Fundamentals
    • System Administration Fundamentals
    • Cloud Computing Fundamentals
    • Security Fundamentals
    • DevOps Fundamentals
    • Supporting Applications and Developers

    The categories will be complementary, with Linux Fundamentals including user and password management, for example, but security going more in-depth on that particular topic.

    In keeping with being accessible for people with entry-level knowledge of the above areas, this certification will be a single exam. That means a user need not wade through dozens of courses and exams to get started in their chosen area of specialty, and the other two certifications are available when the user is ready to move beyond the basics.

    Currently, The Linux Foundation has the following learning path for these exams:

        Introduction to Linux (LFS101) Training
        Essentials of Linux System Administration (LFS201) Training
        Linux Foundation Certified System Administrator (LFCS) Certification
        Linux Networking and Administration (LFS211) Training
        Linux Foundation Certified Engineer (LFCE) Certification
        Linux Security Fundamentals (LFS216) Training

    The LFCA is designed to fit between “Introduction to Linux” and “Essentials of Linux System Administration” to make a pairing of courses and certification exams to carry a system administrator through their career.

    In the midst of business closings and social distancing both increasing the amount of working from home that knowledge workers are involved in, on-site proctored examinations have suffered. This is unfortunate, as there is a spike in the number of people looking to improve their position through certification.

    Since its first exam, the Linux Foundation has offered online exams through a partnership with a proctoring organization. The approach is simple: Webcam, screen-sharing and chat are available to the proctor to monitor the test taker’s activities. Further, the entire session is recorded, so any misunderstandings can be reviewed. The fact that remote proctoring is standard for The Linux Foundation and the organization has been doing it for years hopefully means that proctoring is unobtrusive, and the test-taker can generally just take their test with minimum technical issues.

    The rush to remote proctored testing has created problems for some organizations; having had years to work it out should mean this is not a problem for The Linux Foundation in general and LFCA specifically.

    So if you have friends/coworkers interested in moving in this direction but are unable or unwilling to work through four years of university to get started, point them toward the LFCA. There are a lot of options, but the LFCA has less overhead than many of the other options and will help them settle on whether they are ready to pursue their dream.

    We’ll circle back and do a more in-depth look when the certification is released because there are two parts to a certification—who it benefits and what prospective employers think. We’ll be interested in seeing how this fits with employers’ needs, and to delve into sample questions when they are available.

  • Onboarding Is Science, Not Art

    Onboarding Is Science, Not Art

    Make sure your new team members are getting a well-rounded onboarding

    In Fortnite: Save the World, there is a questline in which a character named The Cloaked Star answers, “Figure it out yourself,” to a variety of questions from a new hero who was a DBA before the end of the world came. It is hilarious because, at that point in the story, the new hero doesn’t have the information to do so.

    Most of us have onboarding of new DevOps employees so simplified it would make us from five years ago cry tears of joy. Give them a container or virtual that is already equipped the way they need and they are ready to be productive, right?

    Welllll … kind of. Let’s stay totally stoked by the idea that pre-made tooling that will run fine on whatever they’re running it on is a huge step, then acknowledge that in the age of DevOps, tribal knowledge is still very much a thing. There is this other repository we hit for X, an alternate API for Y, that system that does not work the way one expects, and so on.

    We need to apply the same science to knowledge transfer that we do to tooling. It is not all self-documenting, it is not all obvious, and that knowledge absolutely needs to be transferred to new employees.

    Some of us have this down pretty well. Far too many of us do not.

    Two solid ways to transfer knowledge are documentation that is both thorough and up to date, and a buddy system. Both have strengths and weaknesses, but both address the parts that are not automated. If you go with documentation, make certain someone is responsible for keeping it up to date in the fast-paced world we find ourselves in. If you adopt the buddy system, make certain your team knows what information is most important to impart. While each buddy will be asked questions, it is the things that might surprise new employees that buddies should be prepared to impart.

    You are building the future “legacy code.” How far in the future that is remains to be seen, but given our rate of change, probably sooner than you think. Even if you use the buddy system, there needs to be some level of documentation included. The number of projects that use something like “Joe’s database designed in his basement” because it had the best caching available but then employee turnover removed everyone with any knowledge of the weak spots in the chosen product, is stunning.

    So make sure your new team members are getting a well-rounded onboarding. Make certain they have the info they need to be productive and avoid pitfalls that your team has already identified. Make their transition easier, and they’ll be productive sooner. And hey, you might avoid that big bug caused by someone’s lack of information.

    Whatever you do, no matter their level, make sure they know the important bits. And keep kicking rear, now with new team members, equipped to help you along.

    And don’t be The Cloaked Star. Help a new employee out.

  • The Pros and Cons of Configuration-as-Code

    The Pros and Cons of Configuration-as-Code

    Configuration-as-code, or “GitOps” as many are now calling it, is a simple but astounding idea: Standardize configuration, check it into version control; no need for each Operations/DevOps person to develop their own set of scripts to get the job done. Configuring or reconfiguring an environment is as simple as checkout and run, assuming everything in the configuration is tested. For a project or a team, this is a serious time-saver and limits the amount of human error introduced into our systems. For projects rolling out code frequently, it is pretty much required.

    But like everything, it has its limits. I worked with a product management engineer once who had concocted a script that had tens of thousands of lines of code and was every bit as complex as the product he used it to manage. His scenario was the typical precursor to configuration-as-code: He would go show off the product and whatever functionality he needed could be managed through his script. The problem was maintenance; as the script was central to how he did his job, changes to it were a big deal. And even though he was the only one maintaining it, he did get burned a couple of times by changes he’d made, because it was that complex.

    When our configuration-as-code starts to cross teams, or we take it to the enterprise level, we risk finding its limitations. Our environments are complex, very complex. And we can do a lot with configuration-as-code, but throwing everything plus the kitchen sink into that system will create issues we need to overcome.

    A single project is constrained. We know what we need to do to spin up the environment. We can identify weak points and build checks and balances around them.

    The enterprise environment is not a project. Agile projects and DevOps thrive because they have a stable working environment to run on. The volume of change in things such as storage allocation, routing and VLANs is not the volume of change that an agile project sees. Good documentation is as solid a solution as configuration as code for something that changes rarely and the size/number of scripts and configurations that need to be maintained is smaller.

    There is a good use case for corporatewide configuration-as-code for recovery purposes. It is far easier to restore the environment in a disaster if you have all configuration and the scripts to install that configuration available. Unfortunately, that use case is minimized by the march of technological change. During the one total destruction restoration I’ve been involved in (was in charge of), many of our configurations were no longer usable because the products we were running in production at the time of the disaster were not the products we could purchase for the rebuild. We ended up with a shiny new data center from building to servers, but it still involved a lot of manual configuration. So while the use case is good, it is not a panacea unless you are constantly replacing your infrastructure with the newest products, limiting the usefulness of this use case also.

    Some organizations have already moved to configuration-as-code via methodologies such as chaos engineering, but most of us are just starting to consider moving configuration-as-code beyond the project level. When the (admittedly large) step of truly moving infrastructure management toward configuration-as-code, start with inventorying what your operations team does on a daily basis. There are a ton of things—the famous low-hanging fruit—that can be boiled down to scripts or standardized configurations and checked into version control. This gives a central point of truth for how things are configured, tracks changes for when things go wrong, and shortens startup time for new DevOps/Operations staff by giving them a repository of tools to review and work with. When working in the cloud, everything is scriptable, so make that side of operations is completely configuration-as-code. In the data center, you end up with diminishing returns after daily tasks are automated, but assigning someone to codify all possible changes is a worthy goal, as long as the work is prioritized to make the most frequent or error-prone configuration issues a priority. Once the focus is down to hardware, the rate of return for those invested hours is going to go down a bit, since specialized hardware is full of proprietary or market-focused config info. To draw on a comparison I’m familiar with: An F5 ADC has very specific use cases that must be reconfigured centrally, whereas restoring your Kubernetes cluster will generally require less focused configuration and teams spinning up containers will take it from there. Not that Kubernetes configuration is simple; but each project handles the details they require while corporate-shared resources tend to need all of the details handled by the infrastructure team.

    We can discuss this at length, and I have with others who think differently, but small tools are better. Do one thing, and do it well—the original Linux mantra—then make scripts that call the small ones. This contains changes over time and doesn’t create dependencies or touches on things that might not need it in a given situation.

    And keep rocking it. All configuration-as-code does is reduce manhours and opportunity for human error, so embrace it, but be controlled about it. Don’t throw everything in there and then forget about it. Make a plan, make a maintenance plan. Iterate. Improve. And use that extra time to solve user problems, which is why we’re here in the first place.

  • DevSecOps and Passwords: Lessening the Burden

    DevSecOps and Passwords: Lessening the Burden

    The point of DevOps is to make IT more responsive (and hopefully more stable, but we have evidence that is not always the case). The point of DevSecOps (so far) is to make security more responsive and to build security right into the stack. These are both worthy goals. As we all have pointed out repeatedly, an already overworked security team needs DevSecOps to keep up with the increased rate of change that a more responsive IT department represents.

    And yet, while we’re automating and simplifying things, the security end of DevOps is doing little for end users. Automating protection of App X is good for the App X team and good for the company, but it means little to the ultimate users of security. We all know security is a burden that users bear in the hopes of protecting the company, but no one is a fan. So maybe you didn’t have self-service password management, and now you do. Great! That helps end users. But most of us were already implementing self-service in an attempt to lift some of the load off of the security and help desk teams by the time DevOps took hold.

    Maybe you’ve implemented SSO as a result of DevOps. Again, good move! Employees spend less time remembering passwords and more time being productive. There’s less stress, too. But again, that was already well along, DevOps or not. And you know what? Employees are still making passwords they can remember, writing down passwords, re-using the same complex password, etc. To make it more difficult to hack, IT has come up with all sorts of strange rules such as, “Password digits, when summed, must be divisible by the square root of pi …” but that does little if the same password is used for LinkedIn (because it is “work-related”) and LinkedIn gets hacked again. And don’t fool yourselves, I know tech-savvy people who do this. Non-technical people are doing it a lot if techies are doing it.

    Which brings us to my suggestion: Your DevSecOps plan should include a password manager budget. There are a ton of them available, they’re cheap enough no matter which one you choose and they will generate complex passwords and then remember them for the user. Give employees password managers, train them a bit on their use, and reduce the chances that their password is the same as the one they used at pleasehackme.com. The investment is small—a few dollars per user for most managers—and choices include vaults that are local or hosted and the ability to set password-generation parameters.

    The only problem with a service-based password manager is ownership of the account after the employee leaves the company, but a policy can clearly delineate when responsibility transfers to the user and arrange for removing them from systems at the same time.

    More secure, less user hassle (true single sign-on, since they only need to know the password to the password management tool) and low cost—this is the heart of DevSecOps. And the investment is small. A bit of cash, a bit of training. What’s not to love?

  • Documenting: Don’t Build Your Replacement’s Nightmare

    Documenting: Don’t Build Your Replacement’s Nightmare

    Want to reduce headaches tomorrow? Document everything today

    About once a year I feel the need to remind you that you are creating technical debt. DevOps is good stuff, and most shops got over the “whatever a given project team decides” multiplication of services pretty quickly, but every line of DevOps code you write is creating technical debt.

    There is no way around this fact. You are writing code (scripts, if you prefer) and developing configs that make assumptions. Those assumptions will change—products will get incompatible upgrades, vendors will go under, libraries will become outdated … the list goes on.

    We have a long history with technical debt. DevOps partially grew out of the need to spend less time maintaining and more time releasing features, and yet we’re re-creating technical debt; in some cases at a much larger scale. It is unavoidable but can be managed. Keep a reference list of assumptions—“This app assumes environment contains X,” or “This app supports Cisco Network APIs version X.Y”—so you know where to look when things inevitably go wrong. The list conveniently doubles as a handy reference to what needs updating when you find out your middleware vendor is going under or that MySQL is out and MariaDB is in.

    The more cutting-edge tools you use, the more likely that you are baking future problems into your environment. This is not advice to avoid cutting-edge; it is advice to recognize reality and keep closer track as you move new things into your environment that might be half-baked or whose market is destined for consolidation.

    Our IT environment has become super-complex. DevOps helps us manage that complexity but discourages adequate documentation. We’ve all heard, “It’s self-documenting” (no, it’s not) or “There is basic documentation” (basic almost always means, “We scratched some notes when it was first set up”… and haven’t touched them since). Don’t do that. Make sure that the new guy after all of you have left can figure out what is going on in this highly complex environment. It is not up to new people to be super-learners with mind-reading skills; it is up to implementors to make certain new people can figure out what is going on without wasting 5,000 hours pawing through scripts.

    It does not take much time to document dependencies. It doesn’t take much more to document relationships. Future you will be grateful when they walk in and need to come up to speed in a short amount of time. Because it doesn’t matter how automated it is when the automation breaks—it matters how quickly staff on hand can fix it and get things going smoothly again.

    You’re keeping the world running. Take a second and make sure the next person can, too. And keep rocking it.

  • But Wait, There’s More! The Trouble With DevOps Tools

    But Wait, There’s More! The Trouble With DevOps Tools

    DevOps tools today increasingly are no longer out-of-the-box ready for use. And that’s an aggravating issue

    Imagine you bought a new power tool—a sweet battery-powered drill/screwdriver that did everything you needed.

    Imagine it looked as though it ran on its own, but when you got it out of the box, you discovered that it didn’t come with a battery.

    Imagine you got the battery, and only after the battery was installed you discovered that it needed a specialized wrench/chuck to install and remove bits. And you had to go buy that.

    Imagine you got the chuck and were all set to go, only to discover that it would slip and normal drill bits wouldn’t turn unless you had a specialized adapter.

    toolsThat’s what our tools environment in DevOps has become. And we need to demand better.

    If your software requires X and Y and Z to be useful—whether you are commercial or open source, whether X and Y and Z are a million bucks or zero—state it before install.

    We’re setting up an entirely new environment, stretching things a bit for an internal project, and honestly, the amount of “Oh yeah, this is required, too” that we’re finding out after install is insane. We have done our research; we should not be getting surprised by “But wait! There’s more!” type installs.

    For an industry that insists on plug-and-play for all OSes and every odd bit of hardware ever dreamed up, we sure let software steamroll us. Vendors need to be clear about what will be required to run the tool and what will be required to run it effectively before we waste our time. Because there are some dependencies we’re not willing to accept, for usability reasons and/or security reasons. And if, under the covers, the tool does exactly the same thing as another tool … then what, exactly, is the value-add? It better be good.

    We’re just one DevOps group—and a small one at that—but we have decided for this project not to use tools that dupe us into thinking they’re usable out of the box, then immediately require us to install something else. Or at least we’re trying. This has gotten so prevalent that in a couple of areas, we’re struggling. After all, the software still has to be built.

    It’s a good thing drill manufacturers don’t think that way. Maybe there is hope.

    Meanwhile, we’ll keep trying to do what you all are doing: kicking rear and keeping systems running.

  • Knowledge Is Power

    Knowledge Is Power

    I was doing some research over the weekend, and it became abundantly clear that you can find an Ansible script to do pretty much any repetitive task in your infrastructure. This is good news for Ansible users, and the pool of available scripts will, no doubt, continue to grow as critical mass effect kicks in.

    My concern is for those that find a script that does what they want/need, and drop it into their infrastructure. This approach may be a good short-term fix, but is full of potential peril. Even a cursory read of a complex script is insufficient.

    We all know the watchwords of the day are “speed” and “agility,” but take some time to test and get to know those scripts. When (not if) they fail, you’re going to want a clue of what is going on in there to track it down, and you certainly want to understand what is being pulled into your infrastructure. While the community is likely to catch malicious scripts relatively quickly, the inclusion of questionable sources and cases where your infrastructure causes a script to have issues are more likely to make it to your desk.

    The easiest validation cycle is to:

    • Read it for understanding. Know what it is doing, why it is doing it and what it is including.
    • Run it in an isolated environment. A pod, a VPN, whatever, just somewhere that you can control ingress and egress.
    • Scan the results as you would anything in production. Make certain sources are valid for your organization, make certain inclusions are safe and validate that any vulnerabilities in the resulting systems are known and acceptable to your organization.
    • If you can, do runtime testing upon it to catch any real-time changes. Most “changes after install” type of software is either well known or not included, so this step is more peace of mind than mandatory.

    If your thought was “takes too long, not necessary,” the risk is on you. But on the list of weaknesses in DevOps, that mentality is definitely in the top three. “Takes too long” is an organizational concern that is different everywhere. “Not necessary” is only true until it isn’t. As an IT professional, having at least a passing familiarity with the environments you are creating is part of the job. Fail to do it at your own risk.

    Much like there are trusted sources for the various types of repo we access, trusted playbook developers are a list you can curate. I know of a few that, in a pinch, I’d be willing to drop a script from into my architecture with only a cursory review, then give it a more thorough going over later when there is more time. But I know that because I’ve read their scripts, and I get that they are also careful.

    While I have Ansible scripts on my mind because that’s what I was looking into, this advice applies to any and all automation scripts that pull from outside sources. Python and Node do a ton of this too, but only highly complex projects pull from a lot of disparate sources. Ansible and other infrastructure automation tools pull from a variety of sources almost by definition (because infrastructure is complex), so they deserve more attention.

    In the end, automating all the things changes nothing about responsibility. So, take the time to make certain you know what is going on in your environment, and why. If you are really motivated, take the time to sweep through and remove any references you are certain you don’t need — but nested dependencies make the ROI on this type of work minimal, in my experience.

    Meanwhile, you are keeping a ton of servers running and users happy. Keep rocking it, and keep looking for solutions like automation scripts to make you more responsive — just don’t relinquish your authority for the datacenter to random submitter on the internet — because responsibility will still be yours.

  • Review, Review, Review

    Review, Review, Review

    Ossification is an issue in IT, and always has been. It is borne largely out of staff and management knowing what to expect from existing software/infrastructure, and the deep product knowledge that allows teams to make existing systems do what is needed. But this becomes a series of rocks that the ship of progress smacks into.

    Contrary to popular belief, DevOps and other new methodologies are not immune to this problem. Quite the contrary, we’ve spoken with people that say “We’re a Jenkins shop,” which sounds exactly like “We’re an Oracle shop” of a decade ago. The wave just ushered in new tools, it didn’t change tendencies. OK, it did more than usher in new tools, but it didn’t change tendencies.

    So, it’s time for a friendly reminder: Review what you are doing, review cloud usage, review tools in the toolchain, review methodologies, review security posture (which has suffered in this environment for far too many shops) and review customer satisfaction.

    Many organizations used the push of Agile and DevOps to update stagnant parts of the infrastructure and development toolsets. Don’t let them become stagnant again. Just as you don’t want change for the sake of change, you don’t want to stay with one tool or methodology just because it is comfortable.

    This isn’t some easy task that I ⁠— or anyone else, no matter how much they try to tell you differently ⁠— can dictate to an organization. There isn’t a handy list, because no two organizations are exactly the same. Not in toolsets, not in culture and not in appetite for risk. The only thing we can offer is guidance to look closely at where you are, what’s available in a given area ⁠— Application Release Orchestration, for example ⁠— and evaluate the available tools just like you were making a fresh choice and did not have a tool in place. Then do the same for processes. Is this the best process for what we’re trying to achieve? Are there tweaks we can make to it? Should we completely change the process to take in changes in environment that have happened since we adopted this process?

    Finally, if you don’t have someone responsible for monitoring cloud costs, appoint someone. Get them the tools, help them understand where the money is going and what cloud-based server usage is. Could these servers be spun down outside of business hours? Could we follow the sun better by staggering cloud data centers in which cloud servers are active, rather than hosting them always in one location or leaving them on 24/7 in multiple locations? Is one team using a cloud feature no one else is? If so, how does it apply, and are the other teams not using it because they don’t need it, or is there some other solution they’ve adopted?

    Basically, question everything. We’re launching a new project that will be a simple dynamic app. Make a few selections from a limited list, get a summary of what you should do and why. It is all really quite simple. Should we host in a web page or make it a standalone mobile app? If mobile app, what tools for development and what targets? Integration of Git — should it be local or on one of the big providers? Do we want any team member to trigger builds on commit or do we want someone managing branch merging before-hand?

    These are all things we already do on other projects, but we’re digging in and seeing what could be better. What could we improve, and what would the rewards for that improvement be? It takes a bit of time and some discussion. We’re confident we’ll burn through it in the next day or so and come up with a plan for development and architecture that best suits the app, but is informed by where we’re at now. It’s been a fun exercise, to be honest. If nothing else, it gave us a strong reminder that any given cloud provider’s “simple” pricing is not that simple, and we need to pay more attention to app characteristics versus fees.

    You’ve rocked it this far, even kept the lights on during the great shift to working from home. Don’t stop improving the environment though, find innovative ways to make it better and keep clipping along.

  • No Excuses: Lock Down RDP

    No Excuses: Lock Down RDP

    We’ve been at the “everyone working remote” thing for months now, and still there are horror stories of people being attacked via Remote Desktop Protocol (RDP). Early on, it was understandable if not acceptable—but we’re not early on any more.

    During the rush of doing whatever it took to help people work from home, exposed RDP was just one of many stories we heard from systems operators, helpdesk and systems admins. Stories of people carrying desktops home, or thinking that tethering to their phone was a “good enough” internet solution predominated. But those issues are clearing up, and they’re specific to a company or a subset of employees. RDP seems to be more persistent.

    According to Shodan (article last updated end of March 2020), RDP use has spiked—and unprotected RDP use has spiked. Security by obscuring (exposing RDP on non-standard ports) has also gone up, and is not effective versus a determined attack. If your goal is to be secure, history has shown very well that obscurity does not at all work.

    Gil Rapaport wrote a great article about this over on our sister site, Security Boulevard. The only thing I would add is to put it on a Virtual  Private Network (VPN) or RD Gateway with 2FA, if you can. Some organizations still don’t have access to VPN services, but if you do, and RDP isn’t running through them, get to cracking.

    Some areas of the world are coming out of lockdown, but the amount of remote work will not drop to the levels it once was. Not soon, probably not ever. The only thing that was holding up remote work for a wide selection of office jobs was the belief that present was better. Some organizations, via forced remote, have learned that the work can indeed be done by people not warming a seat at HQ.

    So don’t put it off. Make certain you are protected. Sleep soundly at night, knowing that the easy vector for attackers right now is protected for your organization. And keep kicking remote rear. Systems are still running, even with the wild changes that came suddenly, and that’s all you–whether your organization recognizes you or not.