devopsy.kinsta.cloud

Category: Enterprise DevOps

  • Kyndryl Aligns with Microsoft in the Cloud

    Kyndryl Aligns with Microsoft in the Cloud

    Following its spinout from IBM, Kyndryl has allied with Microsoft to increase the scope of IT services it provides on the Azure public cloud.

    Stephen Leonard, leader of global alliances and partnerships for Kyndryl, said that as part of this initiative, Kyndryl will be expanding the scope of the services it provides to both organizations building and deploying applications as well as those looking to migrate existing applications to the Microsoft Azure cloud.

    Over time, Kyndryl will expand its cloud services to include additional public clouds as part of a larger effort to enable organizations to embrace a hybrid cloud approach to enterprise IT that spans multiple on-premises and cloud computing environments.

    In the meantime, as part of the first partnership established by Kyndryl, the two companies are committing to providing services to address data modernization and governance, security and application workloads infused with artificial intelligence (AI) capabilities. The two companies also plan to establish a co-innovation lab. Microsoft is also committing to establishing a Kyndryl University for Microsoft.

    Leonard said given the overall shortage of IT expertise, more organizations than ever are relying on external IT services to, for example, accelerate application development. Kyndryl plans to work with those organizations to best evaluate where specific classes of application workloads might best be deployed, he noted.

    It’s become increasingly clear that more organizations are attempting to integrate third-party IT services within DevOps workflows. In some cases, that means standardizing on an opinionated DevOps workflow defined by the IT services provider. In other cases, the IT services provider is focused on a specific task such as securely provisioning infrastructure on behalf of developers.

    Regardless of the approach, however, DevOps workflows are evolving into a team sport made up of both internal and external IT professionals that are trying to bridge the cultural divide that always exists between different organizations. Which IT services provider organizations ultimately select often has as much to do with how flexible they are from a cultural perspective as much as it does their technical expertise.

    For its part, Microsoft is counting on IT services providers such as Kyndryl to increase the total number of workloads running on its cloud. Now that Kyndryl is independent of IBM, it is free to establish partnerships that previously would have been more complex to navigate.

    Of course, it’s up to each organization to determine the degree to which they prefer to work with a larger IT services provider versus a smaller rival that may specialize in one specific area or that is simply a better fit for a smaller company. The one thing that is certain is that as IT continues to become more complex, it’s not feasible for organizations to hire all the IT talent they require on a full-time basis. Organizations will need to carefully evaluate when it makes more sense to consume certain IT services as a function provided by someone else versus attempting to hire and retain talent to maintain that service themselves.

  • Nvidia/ARM Wavering | Google Outage Outrage | Backblaze IPO on Fire

    Nvidia/ARM Wavering | Google Outage Outrage | Backblaze IPO on Fire

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    Nvidia Can’t be Allowed to Buy Arm

    First up this week: Nvidia’s faltering attempt to buy Arm: Regulators around the world are alarmed by the idea. And now the UK—the legal home of the company—has taken the next step, which could nix the deal.

    Analysis: RISCy home for ARM

    This matters because ownership of the ARM ISA and the core designs would be a huge strategic stick to beat Nvidia’s competition with. Arm doesn’t make chips—it licenses technology to Nvidia and many of its competitors—AMD, Apple, Broadcom, Samsung and Qualcomm, to name but five. So this deal really shouldn’t go through, regardless of Nvidia’s fluffy PR reassurances.

    ARM chips aren’t just found in 99% of smartphones, but are an increasing fixture in the datacenter—especially where “performance per Watt” is important. They’re even found in exascale iron, including the Riken Fugaku, the world’s fastest supercomputer.

    Helen Thomas: Nvidia faces tough Arm wrestle

    It is one thing for a regulator to give your deal a thorough going-over. It is another for it to imply publicly that your assurances [aren’t] credible. … Nvidia got both from the UK’s Competition and Markets Authority. … The CMA’s full phase 1 report doesn’t make for easy reading.
    …
    The transaction, which was announced in September 2020, is bogged down in regulatory reviews around the world: … Arm’s “neutral” business model has always made it a difficult acquisition. … Arm licenses its designs to other chipmakers, providing scope to restrict rivals’ access.
    …
    Arm was a political talking point in 2016, when its sale to SoftBank … was waved through with embarrassing ease. There is now a much more muscular political take on UK national security concerns with a particular focus on tech.


    But if it’s owned by a Japanese company, why is the UK getting involved? Glad you asked—Aighearach answers:

    What did they buy? Shares of a UK company. … Then it is a UK company! … Buying shares is buying ownership, but it doesn’t move the company to another jurisdiction.
    …
    This blathering about “sold to a Japanese firm” is getting old.


    Nvidia claims, “The transaction will … boost competition and innovation.” Falmari finds that hilarious:

    Did Nvidia’s spokesperson previously work as a stand-up comedian? Because as jokes go, “boost competition” is a whopper. [Do you think] that’s what a company like Nvidia wants—to boost the competition?

    Google Cloud Outage has Ripple Effect

    Google’s load balancers all went offline for two hours on Tuesday. Aside from the knock-on Google Cloud failures, DevOps tools such as BigCommerce, Datadog, Deno, Discord, Egnyte, Evernote, Linear, Netlify and Toggl were reported to be affected. But perhaps the worst impact was felt by the poor, unfortunate users of Pokémon GO.

    Analysis: Paralysis

    In theory, putting all your eggs in one basket is okay—so long as that basket can reliably stop your eggs smashing on the pavement. But this isn’t the first time we’ve seen Google Cloud issues such as this; I can’t imagine it’s the last.

    Steven J. Vaughan-Nichols: Google glitch triggers major internet outage

    The result? Many of us have seen 404 errors when trying to get to some of our favorite web pages. … There appears to be more major sites having problems than not.
    …
    The specific problem seems to be with the GCP load balancers. These, working with Google’s Cloud Delivery Network (CDN) [are] designed to stop website failures and Distributed Denial of Service (DDoS) attacks by putting your website behind a single anycast IP.


    Are you feeling a touch of déjà vu? jeffbee, too:

    This is a sort of repetitive outage for Google. They’ve wiped out the GSLB configs before.

    A year ago there was also that big outage where they blanked out the whole Gmail delivery config and started rejecting all mail. … Config safety is not their strong suit.


    The #3 IaaS provider won’t get to #2 if it keeps this up. Here’s u/totopapa:

    I’m trying to convince my company to switch from AWS to GCP but an outage like this one is definitely a showstopper.

    Backblaze IPO Burns Brightly

    Backblaze is now NAS:BLZE. In its first week, the stock jumped 60%.

    Backblaze is well known for its tiny-margin consumer backup product and its open-source “storage pod” hardware—but is less well known for its DevOps-focused “B2” cloud storage service.

    Analysis: Can it Avoid Death-by-Packrats?

    Backblaze is trying to grow this newer DevOps business, without losing its classic consumer chops. But many of its former competitors have essentially dumped their “unlimited” offerings—it’s the old tragedy-of-the-commons problem.

    Jordan Novet: Backblaze [goes] public on small valuation and minimal funding

    In an era of blockbuster market debuts preceded by astronomical private financing rounds, Backblaze … a cloud service for backing up data … is a throwback to a time when sub-billion-dollar companies routinely went public to raise capital and build their profile … to win the trust of customers.
    …
    Backblaze shares climbed 24% in their debut on Thursday and another 12% on Friday, lifting the price to $22.31 and giving the company a market cap of around $650 million. … By the standards of its peers, Backblaze is a micro-cap.
    …
    One of Backblaze’s competitors is Amazon, the world’s fourth most valuable U.S. company. … Backblaze currently stores two exabytes of data — that’s 2 million terabytes — and has a half-million paying customers.


    Donovan Jones: Backblaze Targets $100 Million IPO

    The firm provides cloud-based storage services to small businesses and individuals worldwide. BLZE has grown revenue and gross profit impressively through the 2020 pandemic period and the IPO appears fairly valued, so is worth a close look.
    …
    The firm’s financials show strong revenue and gross profit growth. … The firm’s dollar-based net revenue retention rate for 2020 was 114%—a solid result. … The market opportunity for providing cloud-based storage services to small businesses and individuals is large and expected to grow at a material rate. … As for valuation, compared to partial competitor Dropbox, management is seeking the same Enterprise Value/Revenue multiple. [However] the combined revenue growth rate and EBITDA percentage rate … was 3% as of June 30, 2021, so the firm needs improvement in this regard.


    But there’s a long way to go, opines John Kozubik (aka rsync):

    Of the ~500,000 customers, 70,000 are B2 customers and 430,000 are “backup” customers—which are flat-rate “unlimited” users. … Roughly 85% of the user-base are paying $7/mo for “unlimited backup.”
    …
    These flat-rate users are very fickle and price-sensitive. … Roughly 85% of the user-base are paying $7/mo for “unlimited backup.” … It’s a fools errand to pursue this market and I assume the pivot to B2 was to build something more sustainable with a much higher price and usage ceiling.

    The Moral of the Story: Success is a journey, not a destination


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Kristin Wilson (via Unsplash)

  • Cortex Expands Microservices Catalog to Improve Collaboration

    Cortex Expands Microservices Catalog to Improve Collaboration

    Cortex, a provider of a catalog for tracking the ownership of microservices, today announced it has added a service creation capability that enables developers to use templates to scaffold new services in five minutes.

    In addition, the company has added a Cortex Teams offering that improves collaboration across teams of developers working on interdependent microservices.

    Fresh from raising an additional $2.5 million in funding, Cortex claimed it has increased its customer base by a factor of 10 since initially launching last May. Those customers include Rappi, Grammarly and 8×8.

    Cortex CTO Ganesh Datta said the company’s microservices catalog makes it simpler for teams of developers, DevOps engineers and security professionals to collaborate at a time when software development is becoming more complex. While microservices-based applications are easier to update and generally more resilient than monolithic applications, composing an application based on interdependent microservices created by different development teams can be more challenging to build and maintain.

    In some cases, organizations can even discover that different teams within their organization have built redundant microservices. Part of the reason that occurs is many IT teams still rely on spreadsheets to keep track of what microservices are being built and which ones are already deployed in a production environment.

    Cortex, in contrast, automatically imports data residing in developer tools to surface who within an organization is responsible for maintaining which microservices. A Cortex dashboard also enables DevOps teams to see which developers are on-call in the event of an unexpected incident as well as monitor and track the latency rate microservices are experiencing and which vulnerabilities have yet to be remediated.

    At the core of the platform is Cortex Query Language (CQL), a domain-specific language that allows DevOps teams to define granular rules that can be applied to, for example, on-call rotations, security vulnerabilities, package versions, service level objectives (SLOs) and the overall health of the microservice. Engineers can employ code in the form of YAML files to set reliability standards across teams and types of services.

    Grades can also be assigned to specific microservices based on associated metrics. That approach allows DevOps teams to apply gamification techniques using a scorecard to encourage developers to continuously improve their microservices. That capability makes it apparent to other developers which microservices can be most reliably employed.

    It may be a while before microservices-based applications dominate the enterprise landscape. Monolithic applications will be running in production environments well into the next decade. However, many of those monolithic applications are essentially being converted into a larger microservice that other applications can invoke via application programming interfaces (APIs) before being eventually being re-engineered into a set of logically connected microservices.

    The one thing that is clear, however, is that as the number of microservices expands, it becomes more difficult to determine precisely what capabilities are already available to highly distributed application development teams.

  • Are You Feeling the Hiring Crunch?

    Are You Feeling the Hiring Crunch?

    After years of experience, we can now definitively say one thing about information technology: There is always a hiring crunch, and it will always be better soon.

    I first noticed this trend in InfoSec years ago. Everyone was short-staffed and looking for available candidates. Everyone wrote their recruitment text like there was no shortage and they could ask for the moon and everyone was certain that this temporary staffing shortage would end soon.

    It didn’t. It still hasn’t. And it has spread. DevOps in general, development in particular are acting exactly the same these days. And it’s not going to get better.

    So, my advice? If you absolutely must have a person, write your recruitment copy to reflect the actual skills you must have. No bells and whistles, no “We always include…” no “But what if we…” Hire for what you need. Or hire with an eye to training what you need. And offer competitive pay. No one here will argue that IT is underpaid, but when there is a shortage of people, or more are leaving than coming in, then the cost of employees goes up. That’s just the way it is. If you are only hiring the ones you must have, then you can afford to pay them a bit more.

    Meanwhile, invest in even more automation. Let’s face it; again, we can learn from InfoSec. They are doing more with less staff—even if “more” is not “everything that everyone wishes they could”.  They have at least partially conquered their staffing woes by increasing the use of automation. DevSecOps is automating even more of the space, so we really will eventually get over the InfoSec staffing shortage.

    We can do this for all of IT, too—just follow their lead. Invest in automation, take the risk and turn on automated actions. Start small, but get started. Relatively high turnover and lower qualified applicant counts are likely to be the status quo for the foreseeable future, so automating processes as much as possible is not taking away jobs at this point, it is improving response in light of a lack of people for the available jobs.

    So, to sum up:

    • Hire when you can, but be focused. Don’t ask for experience in all three big cloud providers if you’re looking for an Azure person. Don’t ask for fifteen years of experience in Go for that entry-level position. Be realistic, be laser-focused and get the person you need in.
    • Train when you can’t hire. Let’s face it, there are going to be people who are massively familiar with application security but who have never done DevSecOps specifically or never used your DevSecOps solution. Train them—whether they are internal or hired with an eye toward training.
    • For non-critical, long-term needs, explore automation options. That includes both things that are being done manually today and things that are not being done at all. Find a way to get it done in the absence of hiring someone.

    And keep rocking it. You are already getting the job done—I’m just recommending ways to stay on top of it and deliver more while you try to find the additional people you need. A balance of automation and hiring will help you sleep at night and make the rest of the org certain that IT is on the ball.

  • LogicMonitor Introduces Unlimited Log Data Retention

    LogicMonitor Introduces Unlimited Log Data Retention

    LogicMonitor today announced it will provide an option that allows IT organizations to retain an unlimited amount of log data on its software-as-a-service (SaaS) platform.

    Tej Redkar, chief product officer at LogicMonitor, said one of the issues that is holding back advances in observability is the cost of storing log data. LogicMonitor has decided to remove that issue by allowing IT teams to store log data on its LM Logs service for as long as they require, Redkar said.

    That data will also always be readily available as hot storage with no need to wait for data that has been stored offline to be rehydrated, he noted. Instead, that data is readily accessible alongside the metrics and distributed tracing data that LogicMonitor also collects, said Redkar.

    In general, LogicMonitor is making a case for centralizing observability for the entire IT organization via a SaaS platform infused with machine learning algorithms that automatically surface anomalies in real-time based on millions of events captured via log data. The issue many organizations encounter today is silos of decentralized observability data that are more complex to maintain while providing less visibility at a higher total cost, said Redkar.

    The LogicMonitor platform, in contrast, is designed to be equally accessible to, for example, a network operations team as it is to a DevOps team, he noted.

    Not every organization needs to store log data forever, but Redkar said as IT management continues to evolve, more organizations are moving toward storing metrics, traces and logs for longer periods of time. The rise of open source agent software is also making it less costly for organizations to capture that data in the first place.

    It’s not clear to what degree other providers of observability platforms will provide unlimited retention of data, but as the cost of storing data in the cloud continues to drop it’s apparent that billing IT organizations for storage costs is becoming untenable. There is always going to be some cost associated with storage, but it’s not enough to charge extra beyond the core cost of the service. The only thing accomplished by billing separately for storage is discouraging IT teams from storing the data required to both attain observability and better train the machine learning algorithms that surface anomalies.

    Observability, in one form or another, has been a core tenet of DevOps best practices for years. Initially, DevOps teams focused on continuous monitoring as the most effective way to proactively manage application environments. Observability platforms infused with machine learning algorithms make it possible to correlate events so that analytics tools can more easily identify anomalous behavior in real-time. Armed with those insights, it becomes a lot simpler for IT teams to resolve issues faster.

    In fact, there may even come a day when the so-called “war room” meetings that are convened to identify the cause of an IT issue via a painstaking process of elimination are no longer required. In the meantime, however, the total number of IT incidents that actually lead to disruption should steadily decline, even as IT environments become more complex.

  • 4 Keys to Developing Ethical AI

    4 Keys to Developing Ethical AI

    Companies continue to grapple with how to use artificial intelligence (AI) technology in a way that doesn’t compromise privacy and that takes into account other key ethical considerations. It is undeniable that AI is one of the fastest-growing categories today. But if we are unable to build tools that live up to high ethical standards, not only are we doing end users a disservice, but we are failing to deliver on the promise of this technology as a force for good.

    AI and ethics has been one of the hottest topics within the technology community for decades. However, as an industry, we are still finding it hard to put our best foot forward in terms of ensuring ethical use—let alone solving the problems these systems were meant to address. Fortunately, there is one group that could be the guiding force we need: Developers.

    From the ideation process to implementation, the developer community is by far the group with the most intimate knowledge of how AI products actually work; therefore, they can have a tremendous influence on what these tools actually do and how ethical they are—and they need to be empowered to make these calls.

    With that in mind, here are four key questions that companies and their developer teams should ask as they work to build the ethical AI solutions of the future. 

    Is This Transparent in Every Way?

    If a developer ever finds themselves saying, “I don’t know,” when asked about how a product works or how the product is ultimately going to be used, production should stop immediately. The power of automation is tremendous. But given how quickly it can morph and learn, making sure that oversight and explainability are ingrained in its functionality is imperative to ensure ethical use. If this can’t be guaranteed, developers should be empowered to flag their concerns and look for ways to remedy them before continuing production.

    As AI continues to evolve, designing and developing AI systems will likely continue to lead us into areas where no clear legal, social or policy guidelines address many of the potential uses—and misuses. So, now is the time for developers to establish a transparent and clear channel of communication where they can actively raise concerns, discuss and deliberate on ethical dilemmas with diverse stakeholders. This is crucial in reducing or eliminating any negative effects of AI. Otherwise, we will continue to see the current faux pas around AI ethics continue.

    Have we Accounted for Risk or Exceeded Regulatory Boundaries?

    Virtually any big idea has its risks. Unfortunately, when it comes to AI, these risks generally involve unethical use or unintended consequences. The ongoing debate around facial recognition and AI is a perfect example of this. While this use case is intended to improve security and may have other benefits, facial recognition software comes with significant privacy risks and fears over misuse and misapplication. Moreover, because the regulatory landscape around AI is still taking shape, companies can fall into a trap of looking to exploit these gray areas in exchange for being the first to market or gaining a competitive edge. This is why companies that are not using ethical privacy-preserving AI in facial recognition implementations, for example, are having to walk back their products.

    Simply put, acting in this manner can have terrible consequences for the ethics around an AI product. But there is good news; the fix for this mistake is relatively straightforward: Instead of just making sure that products barely comply with regulatory gray areas, make sure they exceed them. Granted, this can lead to increased go-to-market time and significant leg work. However, this is the only way to realistically “future-proof” products against potential unethical use cases.

    Is This System Adaptable?

    Even the best preparation and forethought does not guarantee that a tool will operate exactly as it was intended. This is why it is so important for developers to have the ability to plan for contingencies and build adaptability into their tools so that adjustments can be made periodically if any ethical questions arise. This can be particularly helpful when it comes to addressing bias concerns that may occur as AI products learn and expand.

    Technology is a dynamic and rapidly evolving category. And as such, developers are often working on very compressed timelines. But to get AI right, developers need time to suss out the bigger picture around their products and bake in everything possible to make sure products are set up for ethical use in the short, medium and long term. If they don’t, not only will reputational damage be done to a company, but a company’s entire product pipeline could be stunted indefinitely as products need to be pulled offline entirely.

    Are Developer Teams Actually Prepared to Build Ethically?

    Setting a concrete mission statement that is centered around ethics is a great start for any technology company. But without a more systematic approach to instilling principled behavior in developer teams, companies will struggle to drive tangible results in their pursuit of building ethical products. Additionally, given how complex ethical AI development has become—thanks to innovations in quantum computing, among other things—even the most seasoned developers will have trouble keeping up with the developments and desired standards relating to a company’s ethical AI goals.

    Therefore, beyond just laying out overarching messaging, companies need to codify their ethical intentions by building an entire infrastructure that equips developers with the ongoing training and resources they need to constantly keep ethical standards in mind. If not, confusion will almost certainly ensue, missteps will occur and ethical standards will slip as a result.

    Unfortunately, despite the promise, the AI world has fallen short of meeting the ethical frameworks we need to have in place to both solve the problems of tomorrow while also serving the public good. But this doesn’t have to be the case. By asking these few simple questions, companies can empower developers and build the ethical development infrastructure that is needed for AI to achieve its full potential.

  • Plutora Enhances VSM Platform with Release Insights

    Plutora Enhances VSM Platform with Release Insights

    Plutora has enhanced its value stream management (VSM) platform to provide dashboards that enable DevOps teams to better optimize application delivery.

    Jeff Keyes, vice president of product marketing and strategy for Plutora, said dashboards added to the platform will make it possible for application release managers to leverage predictive analytics to manage application delivery in a more progressive way rather than always having to react to an unexpected event.

    To enable that approach, Plutora has revamped its release insights dashboard that surfaces problem areas and hot spots in addition to adding analytics that track both software builds and larger project initiatives.

    That latter capability is especially critical for executives that currently find it challenging to keep track of multiple software development initiatives, especially those that may have fallen behind schedule without senior leaders being aware of the implications, said Keyes.

    While VSM platforms are typically used by senior IT and business leaders to keep track of how software deadlines are impacting key performance indicators (KPIs) for the business, Keyes said it’s become apparent that application release managers need access to more advanced analytics to optimize application deployment schedules. The Progressive Release Management capability surfaces insights via dashboards in the Plutora platform; that makes it simpler for application delivery managers to optimally orchestrate allocation of resources.

    Release managers, of course, already have access to a wide range of platforms for automating the application delivery process. However, Keyes said, those platforms don’t provide enough insights into how to optimize application delivery across a DevOps workflow made up of disparate tools and platforms. Plutora enables release managers to employ value stream flow metrics and more traditional DevOps metrics to reduce cycle times and improve overall DevOps throughput.

    Keyes said once organizations are provided with that insight, they should also be able to reduce the overall fatigue level many DevOps teams currently experience by removing friction within those workflows. In fact, that fatigue level is a major reason why there is so much turnover in many DevOps teams, he noted.

    It’s not clear how many organizations have a dedicated application release team, but optimizing the software delivery process is a major challenge for almost every DevOps team. While many organizations routinely make use of continuous integration (CI) tools to build applications faster, continuous delivery of applications has proven to be a more elusive goal. This is largely because each platform on which applications are deployed tends to be unique. As a result, many DevOps teams still rely on spreadsheets to manually manage application deployments.

    However, as the pace at which applications are being built continues to accelerate, it’s clear that application delivery is becoming a bottleneck. VSM platforms provide a means of applying analytics to optimize a process that can then be used to make better-informed decisions that might convince more organizations to automate application delivery. That assumes, of course, that the organization had the foresight to invest in a VSM platform capable of surfacing those insights in the first place.

  • Nobl9 Adds Hydrogen Platform to Surface Technical Debt Issues

    Nobl9 Adds Hydrogen Platform to Surface Technical Debt Issues

    Nobl9 today unveiled a Hydrogen platform that makes it simpler for DevOps teams to identify technical debt issues as they manage service level objectives (SLOs) as code.

    Hydrogen enables DevOps teams to define SLOs using a simple wizard or by embedding code within their application as part of a DevOps workflow. That code is then used to automatically collect data from platforms such as Datadog, New Relic, Prometheus, Splunk and others with requiring any additional instrumentation.

    Nobl9 COO Kit Merker said that approach also makes it possible to surface technical debt that inhibits achieving those goals via the Hydrogen platform. Armed with those insights, it then becomes easier for DevOps teams to focus on reducing or eliminating that technical debt because Hydrogen automatically generates a ticket in the Jira project management application—widely used by DevOps teams—to make sure the issue is addressed. Otherwise, in the heat of a DevOps sprint, the issue is likely to be forgotten as the application development project continues to move forward, noted Merker.

    The ability to identify that technical debt is an outgrowth of the ability to manage SLOs as code that is enabled by the core Nobl9 software-as-a-service (SaaS) platform. Hydrogen extends that capability by automating workflows whenever a specific service shows indications that it cannot meet objectives because of a dependency on a legacy platform that needs to be upgraded, added Merker.

    There is also a service health dashboard that organizes SLOs into logical groups that are mapped to features via labels, which Merker noted makes it possible for multiple stakeholders to easily identify which issues affect a particular service most. Hydrogen can also trigger a variety of automated workflows to prompt additional investigations that escalate as services worsen.

    Overall, Hydrogen is designed to provide engineering teams with more accurate context that can be used to prioritize which issues should be addressed first during any technical debt planning cycle, said Merker. Hydrogen can enforce code release policies based on technical debt budgets that are enforced via integrations with continuous integration/continuous delivery (CI/CD) platforms.

    Technical debt generally refers to any code change that is required to optimize a service that is not immediately addressed. The more legacy platforms there are in an IT environment, the higher that technical debt tends to become. At a time when more organizations are building and deploying microservices-based applications, that technical debt, over time, tends to adversely affect applications that are much more dependent on external services than a traditional monolithic application. A single technical debt issue can have a cascading impact on multiple microservices-based applications that would, otherwise, would be difficult to detect.

    The challenge, of course, is that technical debt doesn’t always get the attention it deserves when most of the IT team is focused on rolling out the next great application. However, as application environments become more complex it may only be a matter of time before all the technical debt that is being accumulated becomes unsustainable.

  • Stopping Mobile Fraud With Automation and DevSecOps

    Stopping Mobile Fraud With Automation and DevSecOps

    Mobile fraud is a multibillion-dollar problem that’s only getting worse. One study showed that mobile click fraud alone rose 64% during the first few months of the COVID-19 pandemic. In fact, during the pandemic, about one in five mobile ad clicks were fraudulent. And that’s just one form of mobile fraud.

    Weaponized botnets, mobile malware, mobile Trojans and mobile cheat engines and dynamic instrumentation tool abuse—the methods that fraudsters use to defraud companies and consumers via mobile devices—are constantly expanding in number and scale. 

    Historically, organizations have fought against fraud through monitoring and data analysis. As transactions and data come in, analytics and AI work to discern legitimate events from fraudulent ones. These systems require training and a lot of data to become proficient at identifying fraud, and they can become very accurate. But fraudsters are always changing their tactics to evade these systems, and, as many of us have experienced, false positives are still a big problem. If you’ve travelled enough overseas, likely you’ve had your bank account or credit card frozen until you could call your financial institution to let them know that your purchases were genuine. 

    Another challenge with existing mobile fraud prevention solutions is that they are designed to protect networks and network resources first. As a result, while fraud is occurring and fraud prevention systems are learning, mobile end users are still being affected. Also, when a bad act is identified and a system enforces protections against fraud, these enforcement points typically block network traffic. That protects the network, but the consumer and their mobile app remain at risk. Once a fraudster compromises a mobile app, they may be able to attack through other vectors, even if one network is shut down. For instance, if a consumer unwittingly downloads and launches EventBot to their mobile device thinking it was a legitimate app, it may compromise multiple banking apps. If one bank shuts down its network after detecting fraud, EventBot may have other opportunities on the device.

    DevSecOps

    As this chart shows, once fraud takes place, damage to a brand, customers and the bottom line begins immediately. Even if it is detected and halted, the harm it has already caused can’t be undone. Mobile app publishers and developers need to stop mobile fraud where it starts, and that means implementing mobile fraud prevention inside the app within a DevSecOps framework. By blocking the technical methods used to carry out fraud via mobile apps, organizations can make massive inroads toward stopping fraud before it ever gets to the network level, preventing both customers and organizations from getting scammed.

    Why Traditional Security Approaches Don’t Work in Mobile Apps

    To achieve a multi-layered defense against fraud, a mobile app should include app shielding, anti-tampering, code obfuscation, data encryption (including strings, resources, preferences), jailbreak/rooting prevention and man-in-the-middle prevention (such as certificate pinning or certificate validation). Certainly, there are commercial SDKs and third-party libraries that can provide these functions, with some limitations. But these approaches typically still require extensive development work and introduce additional frameworks or programming language dependencies and incompatibilities. 

    They don’t solve the fundamental problem of implementing fraud prevention inside a mobile app—it’s too expensive and time-consuming.  The root of the problem rests in the way mobile apps are developed, released and updated in a modern agile organization. DevOps depends on processes that are dynamic, automated, agile, integrated, iterative and continuous. Mobile app security has traditionally been very different: Static with manual, line-by-line coding. This way of implementing manual security will never work in an agile delivery model.

    The promise of DevSecOps is built-in security, delivered as a fundamental part of a release process where security is designed into the process at each and every phase of the life cycle. The security model can be delivered and evolved atomically, iteratively and dynamically so that it adapts to fit the app.

    The future of fraud prevention lies in the mobile app, but to make this possible, the industry needs to apply AI and ML to automate fraud prevention and security implementation into mobile apps on time within an agile, fast-moving and iterative app life cycle and release process.

  • Building on Terraform: Evolution, not Revolution

    Building on Terraform: Evolution, not Revolution

    Terraform is widely adopted among both enterprises and service providers who embrace infrastructure-as-code (IaC) as their model for infrastructure provisioning automation in DevOps environments. And while the technology has limitations, innovations seeking to unseat Terraform are highly fragmented and have seen tepid adoption. This points to a market that’s settling on a de-facto standard, so rather than trying to replace Terraform with the next great innovation, infrastructure operators should instead find ways to extend and augment Terraform to suit their unique needs. Let’s explore several of the better-known shortcomings of Terraform, some drivers for the rise in new alternatives to the technology, and what users need to consider when contemplating a different path.

    Without a doubt, Terraform is big. Across the big three public clouds, the number of Terraform provider installs tells the story: At the end of August, there were 457 million provider installs for AWS, and around 72 million each for Azure and GCP. Yes, compared to longer-in-the-tooth infrastructure configuration management tools like Ansible, Chef and Puppet, IaC as a category is still early. But, the trend is clear: IaC is becoming widely used at scale, and Terraform is the IaC platform most enterprises go with.

    It’s also no secret that there are shortcomings to Terraform. A quick search on Reddit and Hackernews unearths the litany of complaints, some more valid than others: 

    • It’s tricky creating a Kubernetes cluster and then adding a resource to it
    • Challenges passing values through a config or dumping a pure representation of an elevated config
    • Cluster deletion is a pain
    • An overall lack of visibility into what’s happening during the provisioning process
    • The need for glue code to drive everything is a drain on developer productivity
    • Lingering challenges with backward compatibility
    • Continuous update challenges and the need to refactor
    • Scaling across multiple workspaces and multiple cloud accounts
    • Rollbacks can be tricky, especially when dealing with immutable infrastructure

    Terraform’s parent company, HashiCorp, of course, steadily works to address these and other pain points. But the whole thing highlights an interesting question: Why is Terraform so popular?

    Terraform is the only IaC solution that meets the needs of developers who want an IaC tool that preserves state, fits well into their CI/CD pipelines, is intent-based and gives them the right level of granularity. Terraform, even with its challenges, is better than the alternatives in these regards.

    Here’s why: 

    Terraform was, from the start, designed for and targeted to developers. It maintains state and supports iteration, making IaC a powerful model for automating infrastructure. PaaS solutions, by contrast, are not robust enough for most developers who do not need or want all of the infrastructure decisions abstracted away. Most cloud management platforms suffer from the least common denominator problem, and even though the technology has improved in this regard, it’s a solution that developers generally abhor for its granularity; it’s geared more to the needs of IT management. Configuration management tools, as an alternative, are not intent-based, so flow becomes too complicated.

    It’s interesting to note that state (maintaining the mapping between the resources that you’ve represented in your definition files and the actual physical resources that are in the cloud provider) was added into Terraform, helping resolve the issue of infrastructure drift. Terraform also realized that developers don’t necessarily need a UI. These tangible examples of the developer-centric design ethos at HashiCorp point to why it is so widely adopted.

    So, what’s driving the interest in new approaches to infrastructure automation? The proliferation of domain-specific use cases and an ever-present need to reduce time-to-value are the leading reasons. Domain-specific automation options are becoming more dominant over OOB solutions because they take too long to get to automation; one tool does not fit all. Domain-specific automation addresses issues such as security automation, AIOps, DevSecOps, security compliance, integration with container scanners—not just generic orchestration. People are looking for value very fast, which can be found only with domain-specific automation—and therefore, fragmentation happens as different solutions emerge to address those cases.

    There are several interesting alternatives to Terraform. Terragrant is used along with Terraform to expand its capabilities to multiple workspaces. It allows you to use the same template across many environments. Pulumi is another alternative that has achieved a measure of adoption and visibility. It provides native code support which addresses the problem with YAML file development complexity. Another more ambitious undertaking is the Crossplane open source project. It takes a Kubernetes-centric approach to creating a universal, cross-cloud control plane. It looks good conceptually, but it’s a fundamentally different approach, and adoption is nascent.

    Turning to one of the Terraform alternatives isn’t necessarily a bad idea, but is fraught with landmines. In the end, this choice is less about technology and more about adoption and ecosystem momentum. Vendor and professional investments in tools, training, and skill sets surrounding Terraform is a formidable moat, and it cannot be easily replicated. Tools that would win over Terraform would have to win adoption, and achieving adoption means delivering all the enterprise-grade features and support that Terraform users currently enjoy. Chef and Puppet are examples of how this could happen. Both were successful technologies, but they were slow to react to the challenge presented by Terraform. Ultimately, these projects suffered from a legacy mindset and were overtaken by Terraform’s more focused, simple solution.

    Most of the issues with Terraform lie in managing it at scale. When you have many workspaces and templates, it becomes hard to manage; it’s also a challenge that templates cannot be generalized among environments, resulting in a lot of template redundancy. Also, Terraform still needs help handling Day 2 operations and visibility into drift detection. 

    When considering ways to extend Terraform to address a broader set of use cases, you should first consider how well things will interoperate across your environment. If you have multiple clouds, multiple service layers and multiple cloud vendors, this is probably the most important factor. Also, investigate any new technology’s ability to support the things about Terraform that are critical to your Day 2 operations. Things like secrets and state come to mind. You’ll want enough management and monitoring in the new solution without it being a new burden to carry. You should be able to use your existing Terraform files, and you should be able to support all the versions of Terraform that you run. Compliance with governance and audit are key, and the new solution should support Terraform for both your on-premises/air-gapped environments and your cloud environments. Ideally, anything you add to Terraform would be open source, eliminating new complications from cross-vendor lock-in.

     

    Terraform use case 

    Architecture for how 8451 evolved its Terraform deployment

    For all its shortcomings, Terraform remains the best option on the market for DevOps organizations, and there are ways to manage it with fewer headaches and without developers wasting so much time writing glue code or reinventing the wheel over and over again. One way this happens is by integrating Terraform with meta-orchestration tools that allow Terraform to become more flexible in managing environments and workspaces at scale. The open source Cloudify project is one example.

    Yevgeniy Brikman gave a talk two years ago at HashiConf that has aged well. At the time, DevOps was still in the Stone Age. Today, it’s probably in the Bronze Age. Those of us building IaC today are being asked to deploy infrastructure that looks like modern cities, but the tools we have to work with remain relatively primitive. What we’re building takes too long to provision and it looks bespoke and temperamental at best or kludgy and fragile at worst.

    It’s a story repeated over and over: Teams try to build their own, homegrown solution first as a wrapper on top of Terraform. Ultimately, they realize that it’s unsustainable as a lack of engineering resources becomes a bottleneck. We need solutions that augment—not replace—what they’ve already built and what works. That’s almost always Terraform. With Terraform, DevOps organizations get a proven IaC option that’s targeted to the needs of developers, with a substantial ecosystem and market uptake. When considering its shortcomings, it’s best to embrace an evolution strategy, and not start from ground zero.