devopsy.kinsta.cloud

Category: Enterprise DevOps

  • Complexity is Still With Us

    We have come a long way and had a lot of improvements over the years, but we still struggle with the ability to simplify things faster than we create complexity. It isn’t enough that each new wave of Really Important Bits™ comes with its own issues and layers and complexity; we also haven’t been cleaning up our previous Really Important Bits™ projects. We haven’t cleaned up the successful ones and we haven’t cleaned up the failures.

    It is time to start. Honestly, we are looking at an extended stretch of ‘Do More With Less,’ and we need to get our houses in order. Want to do more with less? Start by standardizing. Yes, that one project that used Really Important Bits™ (or its companion product Really Cool Tech™) was neat and all, but keeping a technology and its layers and pieces laying around for a single project is not great management. In fact, it isn’t management at all; it is letting things happen. I’d call it chaos, but then we’d have to have a whole conversation about Dungeons & Dragons and, frankly, it is happening in the forum of the overall enterprise architecture so it’s not actually chaos, it is simply excess technical debt.

    DevOps was never meant to be a destination, but on this cross-country road trip, our IT car will get filled with garbage if we don’t take the time to clear out the Really Important Things™ of last year or the last management team or the latest merger. We cannot continue to layer more on without relieving pressure somewhere. AI has the potential to offer some pressure relief in some areas, but we were already behind, so for a variety of reasons, it isn’t the silver bullet we need. Those reasons are simple–it is less secure to leave 50 different ways to do the same thing laying around. AI is resolving some problems and freeing up time, but you’re still wasting time and money on redundant or unused Really Important Things™.

    Some things simply cannot be cleaned up without a massive investment. That’s fine—I hope after years of using DevOps methodologies you have them on a list of things that you’d like to upgrade/replace, but it just isn’t feasible to do so. Focus on the things that you can simplify. The famous low-hanging fruits are fine, just like simplifying things in the early days of DevOps meant that each piece of low-hanging fruit freed up more time to focus on other issues, the same is true now that the low-hanging fruit is generated from DevOps. Or DevOops, in some cases.

    How did I get here, you ask? I was writing a simple overview of Git, CI/CD, DevOps, DevSecOps, GitOps, etc., etc. for a newer technologist, and realized that in between those important parts are a whole lot of tools and technologies we thought were important, but which are no longer terribly useful or necessary. I did her the favor of not including the by-the-wayside technology spaces and product markets. I removed the complexity, you could say.

    And keep rocking it. No matter how far along you are in your IT career, you can look back at all of the things you have learned. Apply that learning to simplifying and improving. Reduce overhead by reducing redundancy and look forward to the next Really Important Thing™ you’ll get to learn about. Keep improving—both your knowledge and the environment. And we’ll keep cheering you on.

  • FAA Ground Stop due to Technical Debt? | Don’t Do DIY Crypto!

    FAA Ground Stop due to Technical Debt? | Don’t Do DIY Crypto!

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The FAA’s NOTAM database gets corrupted, and Threema shows why DIY cryptography is bad. (more…)

  • Are We Delivering?

    Are We Delivering?

    I had an interesting conversation today where I was told, “We do great with CI but terrible at the CD part.” My immediate reaction was to point at all the great CD activity going on in vendor space right now. But as I stopped to ponder, I realized that we’re doing all of that great CD activity to play catch-up.

    DevSecOps has us covered in spades—the toolsets allow for scanning and protection all through the process, from the first line of code to application retirement. I would argue that after being left out in the cold, DevSecOps has done a better job than DevOps at the right side of the equation.

    In DevOps, the various “cloud-native” deployment models (I use scare quotes because they’re not really cloud-native. If you dig into them, nearly every one of them includes Kubernetes in their definition of cloud) are starting to do the job admirably, but in shops starting to automate the entirety of the process, the CI part has been automated and code delivery accelerated for a good long while.

    In short, adoption seems to prove, once again, that the benefits of rapid development are more important than the benefits of offering a stable, repeatable deployment environment. This makes one wonder what, exactly, we should be doing to convince IT and business leaders in orgs smaller than web monsters that security and repeatability both demand the implementation of capable continuous delivery.

    I think we failed the audience when we allowed pundits to blather on and on about deploying thousands of times a second when most applications in most orgs need to deliver at a significantly slower pace. And not only that, but want to deliver at that slower pace, given the “Don’t fix what isn’t broken” rule. Even if the environment is stable, rolling out for a single minor bug fix is possible but it’s not the best use of time—both computer and human.

    But we are catching up. We have things like GitOps and Kubernetes automation catching up with the rate of development; security is already there and test automation is ready to find issues before deployment. It is simply implementation that is behind and, as time goes on, more orgs will get the implementation part nailed.

    And through that process, you’ll be there, rocking it. Step up and make sure it is done right. The entire process is there; you could sleep better at night knowing that it is all in place and you are not likely to get a midnight call. Your org loves stability, you love good nights of sleep … Go out and get it.

  • Survey Sheds Light On Cloud, API Expertise Skills Gap

    Survey Sheds Light On Cloud, API Expertise Skills Gap

    Enterprise IT faces a three-pronged onslaught: Digital transformation, open APIs and security threats. First, most organizations are still navigating ongoing digital transformations and cloud migrations. Second, development teams continue to increase their use of open APIs, leading to sprawl concerns. And third, we’ve witnessed a slew of cybersecurity threats that appear to be growing in sophistication. Tech leaders are attempting to govern this influx of new tools and risks, yet it can be challenging to innovate quickly while ensuring that sensitive data is always protected.

    The Axway 2022 Open Everything Survey shed light on some of the above challenges and attitudes of those invested in API development and API-enabled managed file transfer (MFT). Most notably, the survey found that 86% of respondents feared an expertise gap in the coming years, in which their organization won’t have the skilled software and IT infrastructure talent they needed to innovate. Below, we’ll review other findings from the survey, focusing on the responses from those who indicated they’re invested in API development.

    Cloud Migrations Are Still Underway

    Digital transformations are still underway in the vast majority of organizations. Only 12% reported being wholly digital and leading innovation. The rest were either just beginning transformations, were midway through or were still largely brick-and-mortar. More than one-third (38%) also said their company’s digital makeup is a hybrid of old and new technology.

    The most-adopted computing environment is the public AWS cloud, at 47%. Yet other options trailed close behind, including hybrid (37%), managed cloud services (32%), Azure (28%), private (25%) and Google Cloud Platform (18%). Other reports substantiated how modern IT is becoming increasingly hybrid and multi-cloud. As digital journeys continue to shift and evolve, some organizations are embracing multiple fit-for-purpose clouds.

    Although multi-cloud offers unparalleled flexibility, the primary drawback is security. Just over half (55%) of respondents said that tracking and controlling the flow of personal data into and through the cloud is a key hurdle to a multi-cloud strategy. And 53% reported that dealing with different security requirements for regional instances of cloud infrastructure was a top challenge. This concern is warranted, as companies doing business internationally must carefully navigate many specific regional data regulations.

    Open Data Strategies Abound

    As the Open Everything name indicates, many different sectors are becoming more “open.” The report highlighted the adoption habits of APIs within supply chains, where they are most often used to configure applications, operate and monitor applications and integrate internal data sources. Popular use cases here included payment and tracking systems. APIs also have a significant role to play in the financial world—41% of those working in finance said open banking is an opportunity to discover new business models. But nowhere is the need for more open data more crucial than in health care, which was found to be the area most in need of digital transformation.

    We’d like to think of the API revolution as being driven by API-first startups and SaaS services. While these trends get attention, it’s not the most common use case for APIs. In fact, 44% of the time APIs are used to support IT consulting and system integration requirements. This is followed by providing cloud applications for customers (35%), developing a partner network (35%) and offering cloud services for customers or partners (34%). Although there are certainly business use cases for generating revenue through APIs, they are most often adopted to supplement existing platforms or to glue together internal services and data.

    Most Technology Leaders Fear API Sprawl

    Security is an encompassing concern for all walks of IT, but is especially top-of-mind when opening up data. Respondents said security is the most important concern when creating APIs (30%). This concern is warranted, as attacks on APIs have increased exponentially in recent years. Other top concerns are API adoption or consumption (15%), customer experience (14%), ease of API management (14%) and design and development (11%).

    As APIs and microservices become more abundant within an organization, it’s easier for sprawl to set in. Unbridled technology sprawl can have a deteriorating effect on design standards and lead to unknown shadow APIs that may pose a security vulnerability. As a result, over two-thirds of respondents said the prospect of API sprawl worries them.

    To mitigate sprawl and address other concerns, most companies are investing in tools to help manage the growing number of APIs. The majority (83%) said their budget for API management has either increased or stayed the same over the last two years. For 18%, this budget has risen by over 25%. Operationalizing API life cycle management capabilities can have benefits such as standardizing design style and increasing the discoverability of internal services.

    API Security Must Still Mature

    In a perfect world, we could share data without fear of misuse. But in reality, APIs are a common target for hackers. Publicly exposed endpoints with broken access control or missing object-level authorization can be easily abused for things like privilege escalation and data exfiltration. And although these risks persist, many APIs don’t adopt industry best practices regarding API security.

    Although most APIs use authentication and token schemes, under half (43%) implemented OAuth. Furthermore, only 30% implemented policies with an API gateway. OAuth 2.0 and gateways have become best practices in serving APIs, as there are too many vulnerabilities to rely solely on HTTP Basic Authentication and API keys. As such, we will likely see these adoption rates increase as API security matures.

    As with other cybersecurity endeavors, it’s really about taking a holistic perspective and applying defense-in-depth. Thankfully, most technology professionals understand this point—developing a more holistic approach to security was ranked as the top strategy for protecting personal data. Other strategies, like data encryption, advanced multifactor authentication, data loss prevention tools and highly-secured data transactions will also be necessary to protect cloud-native assets.

    Final Thoughts

    The Axway 2022 Open Everything Survey asked 1,000 IT leaders, architects and developers about their top concerns. Above, we focused on results from technology leaders engrossed in cloud and API development. For more insights and the findings from those using managed file transfer (MFT), readers can pick up a copy behind an email gate here.

  • Southwest Airlines: ‘Shameful’ Technical Debt Bites Back

    Southwest Airlines: ‘Shameful’ Technical Debt Bites Back

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    20 Years of Neglect Led to ‘Meltdown’

    Last month’s débâcle of canceled flights was caused by decades of technical debt. That’s the analysis of Columbia University professor Zeynep Tufekci. (more…)

  • Intel Expands Scope of oneAPI Initiative

    Intel Expands Scope of oneAPI Initiative

    Intel has expanded its effort to drive adoption of a set of open compilers, libraries, analysis and porting tools and frameworks for artificial intelligence (AI) applications, collectively made available via a oneAPI initiative, to include support for its latest generation of processors.

    The oneAPI initiative was launched more than three years ago to create a set of multi-architecture tools for building applications that could be deployed on any CPUs, graphical processor units (GPUs) and field programmable gate arrays (FPGAs). Processors that now support oneAPI include forthcoming 4th Gen Intel Xeon Scalable processors, Intel Xeon CPU Max Series and Intel Data Center GPUs.

    James Reinders, chief evangelist of Intel Software Products, said there is now a clear need for open tools and frameworks that provide equal access to multiple classes of processors. As part of that effort, the latest 2023 version of oneAPI, for example, adds support for plug-ins for Nvidia and AMD GPUs developed by Codeplay, a unit of Intel, to simplify writing SYCL code in a way that makes code portable across additional processor architectures.

    There is also a CUDA-to-SYCL code migration capability that supports more than 100 CUDA application programming interfaces (APIs) using the Intel DPC++ Compatibility Tool based on an open source SYCLomatic project. Intel is also providing tools to identify imbalances at scale using Intel VTune Profiler, and an Intel Advisor tool now provides automated roofline analysis for Intel Data Center GPU Max Series to identify root causes of memory, cache or compute bottlenecks along with insights for optimizing data transfer costs involving CPU-to-GPU offloading.

    As IT continues to evolve, it will become more routine for developers to invoke heterogeneous classes of processors within the same application, but it’s been challenging to convince providers of other types of processors to support the oneAPI initiative, noted Reinders. Processor providers have historically relied on compilers and libraries to lock developers into specific classes of processors.

    However, it’s becoming more common for developers to build applications that invoke multiple types of processors. The Intel oneAPI initiative is intended to simplify access to heterogeneous processors, also known as XPUs. That’s critical, because the pace of processor innovation has accelerated to the point where developers need to be able to invoke additional capabilities without waiting for specific compilers and libraries to be updated by multiple providers. Once oneAPI is more widely adopted, it then becomes possible to significantly lower the cost of application development and deployment, noted Reinders.

    It’s not clear whether the oneAPI initiative will become more widely adopted, either under the auspices of Intel or by some type of industry consortium that provides a neutral venue for building a common set of compilers and libraries. The one thing that is certain is that DevOps teams are vested in trying to standardize compilers and libraries across as many platforms as possible. The challenge, of course, is making sure developers only invoke compilers and libraries that can span multiple classes of processors versus ones that may have been optimized for a specific type of processor architecture.

  • Product Evaluation Discussions

    Product Evaluation Discussions

    Instead of doing a predictions blog like 99% of the punditry, I figured I would do an awareness blog to wrap up 2022. Every year, you deal with a wide variety of people trying to tell you about products that may (or may not) be useful to your IT efforts. Some of the products even make it to the evaluation phase and while a few get installed, a smaller number get used.

    One of the many ways to minimize your time investment when talking about the bewildering array of new DevOps technologies and products is to know where the information you are consuming comes from. Here is my start at a list of information sources, where the information they impart comes from and how to get the most out of these sources.

    Marketing: The marketing team at any organization exists to spread the word about the product’s good points. Most marketing professionals only admit weakness in the underlying product if pressed. This is expected–if the sole purpose of your job is to tell the world why the product is needed, blathering on and on about the weak bits is not going to work out. But we all know there are no perfect products, even before you account for all of the environmental differences we’re installing into. So listen – they will present the product in the best light possible and have useful knowledge, particularly early in the evaluation process.

    Sales: The sales team wants to sell products, but they also are invested in happy customers. A good sales rep in high-tech is building relationships because organizations can create an income stream if the product’s value is realized on their site. This makes them practical. Their goal is definitely to represent the product and company positively, but in a light that admits more weaknesses and offers workarounds for the deal breakers.

    Sales Engineers: A lot of the sales team’s practicality comes from the team’s engineer(s). They’re the ones that have to help you get things installed and running, so they have a vested interest in the organization knowing ahead of time what will require work and what the risks are. Given a choice, talk to the SE alone before signing a contract–not because sales lies, but because the SE is there specifically to note issues that might crop up and how to get around them.

    Crowd Evaluations: Two things to remember about crowd rating sites–the participants are volunteers on the best sites, paid contributors on others. For paid contributors, well, the motivation is pretty clear. For many voluntary participants, the motivation is that they either love or loathe the product. Most of these sites allow vendors to respond to commentary, and some even allow vendors to have evals removed. No one offers a service where glowing vendor reviews can be corrected … keep that in mind. These sites are useful, and a few of them are mostly people that want to help. So find the good ones (I’m not advertising anything here, so no links–sorry) and use them.

    Analysts: Nearly all of the analysts I know–be they GigaOm analysts or a competitor–have experience in the field. They have done it, and most of them continue to keep their hands in the mix somehow. While I have not written (or maintained) a line of code in months, I do attempt to do so regularly. At the individual analyst level, the thing to watch is how long it’s been and how current they are. Some do pure analysis and haven’t touched the toys in a decade or more, and for C-level analysis, this is great … but for implementation recommendations, best to find a different source.

    Analyst Firms: This is where the real issues in analysis creep in. As an enterprise architect, I dealt with an analyst firm that–at the time, at least–would say whatever we wanted them to say. Totally useless in evaluations, but it worked well when convincing our technical advisory committee to approve the right approach. Each firm comes at the problems with its own lens, and being aware of a given firm’s slant is a good step to understanding how to use them–or if you should rely on them at all.

    Press: The age of pure-play press is largely dead, leaving a lot of “Paid for publication” out there. While there is still non-sponsored content being developed (one of the few rules Techstrong Group has for my writing of this blog is that I should avoid mentioning products; for example, making sponsorship they may pursue more about their vendor relationships and how many of you are reading my drivel), the majority of what is published as “articles” is “marketing” and should be treated as such. Reading this content can be useful for early education but not so much for evaluations.

    Internal Champions: Not the mandatory sponsor most projects require, but the person inside that is driving the org to look at a solution. These people are generally key. They know the environment, they know the product, they have ideas on how the product could help the environment, and they have a history in the organization. Whether their ideas about how various products help the company have panned out with other tools, for example. Of course the usual caveats apply… Security people still think you should be doing more to lock out bad actors – that is what they’re paid for, but that influences their evaluation of a new security product and how it fits in the org. Awareness of that type of vested interest can help filter internal champion input.

    There are others, or course, but I’m keeping this blog-length, and most–like MSPs or hosting providers–can be viewed as an extension of one of these (MSPs and hosting providers generally act like extensions of their chosen product’s sales and marketing, for example).

    And keep rocking it. All these people can help you understand how to make the environment better or your role easier. They can’t do it for you; that is your skill. Listen to what they all say and then act in the organization’s best interests. When I’m not a pundit, I tend to use marketing/crowd rating to get a feel for the space, then analysts and SEs to tell me about the space and specific products in it respectively. But figure out what works for your org and be aware of what viewpoints the various actors bring to your learning.

  • Best of 2022: Agile/Scrum is a Failure – Here’s Why

    Best of 2022: Agile/Scrum is a Failure – Here’s Why

    As we close out 2022, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

    Welcome to The Long View—where we peruse the news and strip it to the essentials. Let’s work out what really matters.

    This week, something a bit different: I’m devoting my entire column to Agile and Scrum. They’re increasingly getting a bad reputation, being associated with the worst aspects of toxic workplace culture: Sexism, racism, burnout, micromanagement. (more…)

  • Best of 2022: We Must Kill ‘Dinosaur’ JavaScript | Microsoft Open Sources 3D Emoji

    Best of 2022: We Must Kill ‘Dinosaur’ JavaScript | Microsoft Open Sources 3D Emoji

    As we close out 2022, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: JavaScript is a bloated barrier to progress, and Microsoft’s emoji are on GitHub. (more…)

  • Best of 2022: CI/CD is the New Lock-In

    Best of 2022: CI/CD is the New Lock-In

    As we close out 2022, we at devopsy.kinsta.cloud wanted to highlight the most popular articles of the year. Following is the latest in our series of the Best of 2022.

    Welcome to 2022! Let’s start this year’s blogging off with a bang, and point out that we are in the process of creating the biggest lock-in that we have ever had. And I do mean ever. Maybe the hardware lock-in of the old days where vendor A was incompatible with vendor B was worse, but at least that one we acknowledged when we signed the contract. This one is growing on us as I type.

    Our DevOps environments in 2022 cannot run without what I like to call the “toolchain controller”: GitLab, Github CI/CD, Atlassian, CircleCI, Jenkins—whatever you are using. Like you need some form of faucet or flow control to make running water work, you need the CI/CD environment to make DevOps work.

    You. Can’t. Live. Without. It. And just like any other vendor solution, CI/CD vendors are not going to be in a rush to help you move to a competitor. It won’t be easy or cheap to switch, largely because we are moving everything that might have been on a computer that once had developed code running on it into the toolchain.

    Sooner or later, we’ll back off the kitchen sink definition of “What is DevOps?”, but I predict even then we will have to answer “…A lot of what we do,” because DevOps is, by definition, end-to-end. It is about the process and all the steps from design to production.

    No matter how long or complex the process through which an organization’s CI/CD tool runs, a plan must be in place to replace it. A workable plan, not a check box plan. When Atlassian decided they preferred cloud-based customers, their customer base was shocked. Atlassian was wise enough to leave an option for customers who really had to have an on-site solution, but it is not difficult to imagine them phasing that option out, or to imagine another vendor dropping on-site support altogether. Some of the more competitive offerings in the space—GitLab springs to mind—are online-only already, making that option more appealing to other vendors as it’s now proven viable.

    So, have a plan. This is more important, IMO, than cloud vendor lock-in. Cloud vendors are a small community, and each understands the others, so if push came to shove you could find help moving between the big three cloud vendors. There are dozens of DevOps master controller-type solutions in regular use, and some of them are not at all CI/CD solutions (application release orchestration tools and software infrastructure management tools are two areas doing CI/CD duty at some orgs, for example), so there is not going to be much help moving between them unless you manage to find a very specialized skill set that matches your desired move.

    So, have a plan. Yes, I am indeed repeating myself. Know what you’re going to have to do before you have to do it. Include what to do if the master DevOps controller goes down. Because it will, sooner or later.

    And keep rocking it. Protect your masterworks by protecting what builds them. And keep rolling out the tools your users need. It’s a new year—learn a new skill and use that to turn out even more cool stuff. I haven’t picked a new skill yet, but I’m taking that last bit of advice myself. And again, in the new year, thank you, for all those who don’t know to say it.