devopsy.kinsta.cloud

Category: Enterprise DevOps

  • Testlio Adds Fused Testing to Automate Testing Platform

    Testlio Adds Fused Testing to Automate Testing Platform

    Testlio announced today it is adding fused testing capability to its application testing platform; Testlio is used by IT organizations to manage application testing teams consisting of internal employees and external contractors vetted by Testlio.

    Fused testing describes the combination of automated and manual testing IT organizations use to improve overall productivity. Fused testing strikes a balance between tests that can be automated and those that still need to be performed by a human.

    To achieve that goal, Testlio has partnered with Applitools, BrowserStack, Headspin, Mabl and Sauce Labs. Failure signals and session artifacts from these tools can be pushed to the Testlio platform to streamline the triage of test results. Testlio is making six device slots and 100 hours per month of automated testing processing time available at no additional cost.

    Finally, Testlio has set up an automated testing center of excellence (CoE) as part of an expansion of its professional consulting services it provides to help organizations implement testing best practices.

    Testlio CEO Steve Semelsberger said those integrations will make it simpler to augment human testers and increase the rate at which applications can be tested as many organizations are accelerating the rate at which those applications are being built and deployed.

    As organizations seek to implement continuous testing within the context of a DevOps workflow it’s apparent there will need to be more reliance on automation to keep pace with application development, he added.

    Through its platform, Testlio provides organizations with access to more than 10,000 freelance application testers on demand. That network of testing professionals can create, run, optimize and diagnose automated tests using Appium and Selenium frameworks written in programming languages such as Python, Java and Node.js. Only 3% of the freelance application testers that apply to join the Testlio network have been accepted, noted Semelsberger.

    The degree to which organizations prefer to rely on internal employees versus contractors to test applications will naturally vary. However, it’s difficult to predict what volume of tests will need to be performed—and when they should be performed—to preserve application development timetables. Contractors provide IT organizations with a degree of testing flexibility that is becoming an absolute requirement given the current challenges of hiring and retaining professional application testers, said Semelsberger.

    Application testing is, of course, difficult to apply uniformly no matter how many testers are available. In addition to providing access to experienced freelance testers, Testlio also provides a platform for managing the overall testing process. The challenge is then aligning those testing tasks with the various application development projects the organization has launched.

    Regardless of the approach to testing, the need for it has never been more critical. As organizations embrace a wide range of digital business transformation initiatives, a much higher percentage of the applications are now externally facing. The tolerance for suboptimal application experiences among customers and partners is much lower than that of internal employees. As such, first impressions, like it or not, are going to determine the success and failure of most of those application initiatives.

  • The Life and Times of Feature Flags

    The Life and Times of Feature Flags

    A while back, I had the opportunity to look closely at DevSecOps tools (as the market currently defines them). There was a lot to like, and they are moving forward to a “Find all of the vulnerabilities, all of the time” future … Except there will be new vulnerabilities that aren’t as easy to catch once we get there, and our journey will continue.

    More recently, I’ve had the pleasure of diving into application and API protection, which I can’t (actually, I’m not certain if I can or can’t, so won’t) say too much about at this time, but which are also moving forward in a similar direction, if not in a similar manner.

    And as a developer, feature flags have intrigued me since they first appeared.

    Sing the Sesame Street song with me: “One of these things is not like the others.” Feature flags are … a feature.

    When they first hit the scene, it was literally developers writing #if and #ifdef statements in their code to turn bits on and off at compile time. That reminds me of nothing so much as the Seinfeld episode where they’re explaining to network execs what the show is about. “A developer does his job … that’s a product!”

    The market has come a long way since then. But, honestly, I just don’t see these as stand-alone products. Take a look at the breadth of things most products are doing these days—CI/CD tools, either of the two markets I just mentioned or just about any other market in a similar space. Feature flags are still just about “turn this functionality off, turn it back on”. Newer iterations allow this without rebuilding the entire app. But it’s just not up there with the potpourri that current tools markets offer.

    My take is that these tools will end up as part of a larger toolset. Microsoft, Atlassian and Cloudbees (to name the ones I’ve seen without looking too hard) already implement feature flags, shrinking the available market. And they implement them because it’s a perfect fit with several markets. DevOps tools—which is where each of these vendors implements them—are one of the best market fits.

    Which market will end up with them? I have no idea, and that’s part of the fun as an analyst. The best on-the-surface fit would be DevOps toolchain systems like Atlassian offers, where no matter your toolchain, you can use them to speed acceptance testing or save a release without re-releasing. But I can make valid arguments for other locations too, so I guess you all will decide as options come available—decide with your dollars, that is.

    Watch for it. I’m not at all saying “Avoid feature flag companies because it’s not a real product!” I’m just saying that your feature flag vendor is likely to become part of a larger solution at some point, so make sure you have a plan to move off of it in case the larger solution is something your organization just doesn’t want to deal with.

    I am saying, “If you don’t have a feature flag solution—even a homegrown one—in place, it’s time to put one in place. The ability to say “Oh, yeah, that feature is mucking things up, let’s turn it off, quick!” is huge. For larger sub-projects that take multiple sprints, the ability to have the code right in the stream with regular check-ins, but feature-flagged out of the build, is also huge. No merging or other craziness, just “BuildNewFunction=FALSE” in the source or config file. The more advanced the feature flag system in place the better—the newest ones can turn things on and off with an environment variable from a dashboard. This makes feature flag code inclusion far easier and more intuitive than, “Did we account for that database update not happening?” of hand-rolled solutions. Still not perfect—because this is a journey—but store-bought is now better than anything you’re likely to hand-roll.

    And keep rocking it. Feature flags or not, you’re making systems that power the world—or at least your corner of it—and not getting the recognition you deserve. So once again, thank you. For all those people who don’t even know that they should thank you.

  • How Are API Management and Service Mesh Different?

    How Are API Management and Service Mesh Different?

    The use of APIs and microservices is increasing—a recent study from F5 estimated that the industry is approaching 200 million total APIs. As organizations expand their use of APIs and microservices, they inevitably require some form of service management architecture. There are primarily two major design options for implementing this: An API management system or a service mesh.

    The key distinction between API management and service mesh can be framed using the four cardinal directions. API management is typically adopted for REST services that face external developers—referred to as north-south traffic. On the other hand, service mesh is more commonly used by internal suites of microservices to help control lateral, or east-west, communication. There are more nuances between each, as we’ll cover below, but this is essentially how they’re adopted in practice.

    However, this binary is becoming blurred as collections of enterprise digital applications expand and access to cross-company business domains begins to look and feel more like consuming a third-party SaaS. Furthermore, some suggest that with the right controls, service mesh can cater to external users in the same way API management does. What’s more, service mesh and API management are complementary—they can coexist within the same organization and even be deployed in the same projects.

    I recently chatted with Mark Cheshire, director of product at Red Hat, about the emergence of both API management and service mesh. According to Cheshire, both approaches accomplished similar goals but evolved very differently. Below, we’ll revisit why these two paradigms emerged and compare and contrast them. In a separate article, we’ll consider when it’s best to integrate them both.

    What is API Management?

    API management is most often associated with north-south traffic. By exposing your API to the outside world, you enable other developers to connect it to their systems and leverage your data and functionality like building blocks.

    API management arose in response to the rise of web APIs. API management is a broad concept encompassing many functions required to expose an API endpoint to the outside world. This can include authentication, authorization, request routing, documentation, rate limiting, monitoring, logging, monetization and life cycle management. A key goal is to make integration easier for the developer consumer. Architecturally, API management follows a gateway pattern. Consumers issue requests to the gateway which then performs routing to the underlying services.

    For APIs, the big catalyst was the emergence of Web 2.0, Cheshire explained. Numerous startups began consuming RESTful APIs to connect their web properties and partner ecosystems. Companies also found REST APIs to be a much more straightforward integration approach than SOAP and SOA. They also saw the ease-of-use benefits and adopted API-first designs within their internal infrastructure.

    As of 2021, over 70% of APIs are built for internal use. So, do all these services need API management? It depends. “If you communicate externally, you need API management,” explained Cheshire. But, it’s a grey area for services within an enterprise, he says, though inter-domain traffic often could benefit from API management.

    A diagram of API management capabilities, from 3scale
    A diagram of API management capabilities, from 3scale

    Strengths

    • Can enable north-south traffic
    • Protects the boundary of an application
    • Designed to ease external developer consumption
    • Potential for monetization, subscription plans
    • Typically plug-and-play; easier to support
    • Many solutions on the market
    • Developer experience and docs
    • Analytics and reporting

    Weaknesses

    • May be heavier than service mesh
    • Fewer open source options; vendor lock-in
    • Analysis paralysis in comparing solutions

    What is Service Mesh?

    Service mesh is most often associated with controlling east-west communication between internal services. Service meshes like Linkerd, Istio and Kuma make it easier to reimplement similar logic—such as networking, observability, traffic management, security policies and routing—across many microservices from a single control plane.

    Service mesh was born to address slightly different challenges than API management. “The transition from monolithic to microservices is all about east-west traffic,” Cheshire said. Companies found it slower to innovate monolithic applications and thus broke them up into microservices that communicate via APIs. But these microservices all needed similar operational capabilities, which would be cumbersome to repeat over and over for each service.

    “For one monolithic app blown into microservices, you may not even need a service mesh,” said Cheshire. “However, when you have hundreds or thousands, that’s when applying service mesh becomes critical.”

    Istio service mesh architecture
    Istio service mesh architecture

    Architecturally, service mesh is typically divided into a data plane and a control plane. The service mesh deploys a sidecar proxy that sits alongside each individual service. Istio, for example, utilizes the Envoy proxy. Other meshes like Linkerd use their own proxy.

    Strengths

    • Fits the microservices architecture paradigm well
    • Good for traffic routing; improving resiliency
    • NIST-recommended architecture for achieving zero-trust
    • Many open source options
    • Fits large microservices deployments

    Weaknesses

    • Requires a more hands-on approach
    • Potentially more maintenance to support
    • Unnecessary for a single service
    • Requires filter chain add-ons for north-south abilities

    Comparing API Management and Service Mesh

    API management is predominantly aligned with externalizing APIs to partners. When you do so, maintaining the contract with a consumer is more important to meet SLAs. Thus, API management commonly delivers life cycle management abilities, such as versioning. API management is necessary even if you are exposing a single API, as this service needs to be adequately documented and secured. That being said, these tenants can also be relevant to developers reusing software within an organization—there’s something to be said for adopting a product mindset when dealing with internal APIs.

    On the other hand, service mesh is more aligned with controlling internal networks. Microservices are rarely deployed alone; typically, multiple microservices make up an application. And, many more might exist throughout an organization for unique purposes. Service mesh thus helps maintain consistency throughout large networks of decoupled microservices. Service mesh is also a rapidly growing field, experiencing an explosion of new development and production use. A Cloud Native Computing Foundation (CNCF) report found that service mesh use rose in production by 50% in 2020 alone.

    To put the comparison another way, William Morgan, CEO, Buoyant described how API management is more about business logic, whereas service mesh handles intra-cluster operations:

    “API gateways typically handle ingress concerns, not intra-cluster concerns, and often deal with a certain amount of business logic, e.g. ‘User X is only allowed to make 30 requests a day’. By contrast, a service mesh like Linkerd is focused on operational concerns (e.g. instrument these calls and report their success rate; secure this communication between these pods) and purposefully avoids the mixing of business and operational logic; and focuses on communication between the services on the cluster.”

    Though service mesh and API management achieve slightly different aims, both layers make it easier to reuse web-based services. And, they certainly can be used in conjunction. For example, you could build API-first microservices with service mesh and simultaneously use API management to expose certain domains. In another article, we’ll consider when it’s best to integrate the two technologies.

  • 2022 Predictions: AI, Security and Cloud

    2022 Predictions: AI, Security and Cloud

    It is difficult to find a single area of the technology landscape that isn’t incorporating artificial intelligence (AI) into products and services. As we move into a world where distributed computing is the norm for business operations, AI is a requirement given the increasing level of complexity in computing environments. While there are many different ways AI is penetrating computing, I think that there are a few specific areas where we will see a huge amount of growth in terms of innovation and investment. 

    AIOps will be at the forefront of monitoring and managing distributed computing environments.

    2022 will be a busy year for AIOps vendors and product offerings. Expect that, in 2022, you will see a massive number of large enterprise software companies bringing AIOps offerings to market—either as part of distributed computing management suites or as individual tools. In addition, there will be a flurry of well-funded startups entering the space. One of the first areas where AI models are productized is in the area of AIOps. AIOps provides the ability to apply machine learning models and automation based on data about the patterns and anomalies within IT operations. AIOps is an implementation of observability and helps businesses monitor and manage their IT environment based on collecting and managing data about operational patterns that are expected and correct. It also determines when there is an action that is outside of the norm, and provides either an automatic correction or offers IT operations management the ability to correct errors before they impact business operations. 

    Expect a year of consolidation in advanced analytics, security and solution clouds. 

    We’ve already seen 2022 kick off with a flurry of acquisitions. While we haven’t seen blockbuster buys, we have seen smaller acquisitions by big vendors that need more to sell their existing customer base. One exception is Alteryx, which purchased cloud-native analytics vendor Trifacta for $400 million in cash. In contrast, IBM recently purchased Envizi, an Australian data analytics firm focused on analytics for environmental, social and governance (ESG) requirements. This is just the beginning. We expect there to be a flurry of acquisitions in the hot observability market (solutions focused on analyzing how well the internal states of a system can be inferred from knowledge of its external outputs).

    As security continues to grow in importance, key emerging security vendors also will be hot commodities in the acquisition space. For example, Microsoft has acquired RiskIQ, a security software vendor focused on gathering threat intelligence about a wide range of cyberattacks across multiple public and private clouds. We also predict that many of the large cloud vendors will move aggressively to buy cloud business solutions.  

    Looking Forward

    It is easy to get caught up in the flurry of announcements and acquisitions coming at us fast and furious. However, the reality is that businesses need to know that whatever new shiny objects are introduced, these solutions need to be able to solve real customer problems that impact revenue and growth. It may sound obvious, but I think that vendors need to pay a lot more attention to customer pain. 

  • EEEWW—Green iMessage Bubbles | ‘Great Resignation’ Redux | Faster Firefox 96

    EEEWW—Green iMessage Bubbles | ‘Great Resignation’ Redux | Faster Firefox 96

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: iMessage lock-in lessons, attrition-rate fixes, and more on Mozilla.

    1. ‘Never Date a Green Texter’

    First up this week: The weekend story getting the technorati chattering was about how Apple iMessage denizens pour scorn on Android users—identifying them by their “icky” green chat bubbles. An iPhone user’s message gets a “proper” blue background, y’see.

    Amusing though the story was, the writer seemed to be hell-bent on proving some sort of Apple C-suite-led “conspiracy to lock in.” The reality is far more mundane—but it teaches important lessons for DevOps.

    Analysis: Unintended consequences FTW

    Could your DevOps team make a trivial engineering decision that pays unexpected dividends? While lock-in might be distasteful, seemingly irrelevant design or technical choices can have huge ramifications with users.

    Tim Higgins: Why Apple’s iMessage Is Winning

    [The] pressure to be a part of the blue text group is the product of decisions by Apple executives starting years ago that have … helped to cement the iPhone’s dominance among young smartphone users. [It] encourages people to pay the premium … and remain loyal to its brand.
    …
    Teens and college students said they dread the ostracism that comes with a green text. The social pressure is palpable. … Jeremy Cangiano, who just finished up his MBA, … tried to cash in on it last year by selling his own merchandise that touted, “Never Date a Green Texter.”
    …
    The blue iMessage bubble was born out of a simple engineering need, according to Justin Santamaria, a former Apple engineer who worked on the original feature. … The idea that it would keep users locked in to using Apple devices wasn’t even part of the conversation at the time, he said.


    And then the other shoe dropped—via Ron Amadeo:

    Google took to Twitter this weekend to complain that iMessage is just too darn influential with today’s kids. … The company is hoping this public-shaming campaign will get Apple to change its mind. [But it] probably has the least credibility of any tech company [because it] has released 13 half-hearted messaging products since iMessage launched in 2011.
    …
    Google … seems to have learned nothing from its years of messaging failure. Today … the situation is an incredible mess, and no single Google product is as good as Hangouts was in 2015.


    Ouch. With a sociological perspective, here’s Christina “filmgirlcw” Warren:

    iMessage is incredibly popular with certain demographics. … It’s an in-group signifier—the way teens have always had in-group signifiers (when I was in high school, it was AIM, a Nokia, and wearing Abercrombie). … It’s always fascinating for me to go to other countries and see a ton of iPhone users not using iMessage but something else because that app has become the in-group signifier. … Every teen I know uses Discord more than they use almost anything else.
    …
    It isn’t about lock-in or regulation or technology. It’s about fashion.

    2. How’s your Attrition Rate?

    A study published this week says DevOps orgs and other internet companies rank third for attrition in the 38 industries surveyed. And if you lump that category together with “enterprise software,” it would be in clear first place.

    Analysis: We’re #1! Oh, wait …

    But some companies get this right—how can you be one of them? Culture, innovation, recognition and remote work. It’s definitely not only about pay.

    Donald Sull, Charles Sull and Ben Zweig report their fascinating study: Toxic Culture Is Driving the Great Resignation

    Between April and September 2021, more than 24 million American employees left their jobs. [But] while resignation rates are high on average, they are not uniform … from less than 2% to more than 30%. … More-innovative companies … are experiencing higher attrition rates than their more staid competitors.
    …
    Pay has only a moderate impact on employee turnover. … A toxic corporate culture is by far the strongest predictor of … attrition and is 10 times more important than compensation. … Job insecurity and reorganizations are important predictors of how employees rate a company’s overall culture. … But it is surprising that … the more positively employees talked about innovation at their company, the more likely they were to quit. … Employees are more likely to leave companies that fail to distinguish between high performers and laggards [or] that tolerate underperformance.
    …
    Our analysis identified … actions that managers can take … to reduce attrition. … Lateral career opportunities are 12 times more predictive of employee retention than promotions [and] 2.5 times more powerful [than] compensation. … Company-organized social events, including happy hours, team-building excursions [and] potluck dinners … are a key element of a healthy corporate culture. … Unsurprisingly, when employees discussed remote work options in more positive terms, they were less likely to quit.


    And the flipside of resignation? Hiring replacements. Let’s turn to the equilibrium of Nash Reilly:

    It’s only gotten harder in recent months [and] I think I know why. … You need to offer engineers some combination of these three things to pique their interest: Cool stuff to work on, smart people to work with [and] some degree of repeatability.
    …
    You can’t really prove to an engineer that you can give them things like good management, minimal churn, and decision making authority [so] let them talk to your employees. … Also, figure out a way to let your team “backchannel” to the candidate. [If] your product is not that cool … play up the things you can offer them: smart coworkers, and a stable working environment. … You can afford to be honest about what it is that you do offer to a candidate, [so] pitch them the truth: that you want to do good work, sustainably, for as long as you can—ideally with them on your team.


    Perhaps the smart money is already skating to where the puck will be? Here’s echelon:

    Engineering salaries are … one of the biggest expenses. … Companies love that they can now hire remotely and pay adjusted cost of living. They’ll hire as many workers in middle America as they can, and once enough of the workforce is concentrated outside of expensive cities, you’ll see them stop hiring in SF and NY entirely. [And] startup capital is already beginning to shift to … emerging markets. Employment will eventually head there too.
    …
    Make your money now while the iron is still hot.

    3. Say Something Nice About Firefox

    After accusing Mozilla of jumping the shark last week, perhaps I owe the Firefox folks a positive inclusion this time? The release notes for Firefox 96 certainly sound good.

    Analysis: Improved sound; better perf; stronger anti-CSRF

    Outside the Apple walled garden, Firefox is about the only non-Chromium browser. A browser monoculture can’t be a good thing for anyone—not DevOps nor users.

    Michael Gariffo: Firefox 96 update focuses on noise improvements, main thread efficiency

    On the desktop side of things, Mozilla’s latest release made “significant improvements” to the browser’s built-in [sound] noise suppression and auto-gain-control features. These improvements are aimed at web-based video chat users.
    …
    The second tentpole feature in version 96 focuses on reducing main-thread load, which should help the browser function better on older, slower, or congested systems. [And] a new enforcement setting for cookies … will, by default, apply the “Same-Site=lax” policy. Mozilla claims this will help better defend users against Cross-Site Request Forgery (CSRF) attacks.
    …
    Mozilla also acknowledged that it is working on correcting the performance of detached videos being played in full screen on macOS devices. … No timeframe was mentioned for when it might be available again.


    Okay. Sounds good. But not to Tetch:

    Never mind all that fancy stuff. … Any reduction in crashes?
    …
    FF crashes for me several times a month. … I always take the trouble to fill in the “What were you doing immediately before the crash?” section of the crash submission dialog, and never hear another thing.


    But more efficiency on the main thread? Tell me more. ksec:

    Such an important detail that deserves at least few more sentences of marketing message or explanation. But the whole release note just feels like they couldn’t be bothered anymore.

    The Moral of the Story: A weasel wishing Happy New Year to a chicken harbors no good intention.


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Moritz Knöringer (via Unsplash)

  • How AI Can Improve Software Development

    How AI Can Improve Software Development

    Change is constant and inevitable. While some changes are welcome, others are just a necessary evil. 

    Continuous calls for digitalization combined with a pandemic-prompted emphasis on transforming business operations, force organizations across industries to modernize their legacy systems. The emergence of cloud computing has shrunk organizations’ on-premise IT footprints. However, with 96 of the world’s largest 100 banks, 90% of the world’s largest insurance companies, 23 of the 25 largest retailers in the U.S. and 71% of the Fortune 500 still using mainframe systems, any modernization efforts—resulting in increased risk of downtime or defects—will frequently lead to disastrous consequences. 

    So, while making changes can be expensive, uncertain, and lead to customer loss and/or embarrassment in the marketplace, modern business demands that systems change to continue to deliver value. The problem with mainframe modernization, however, is that today’s code search tools, linters and program analysis tools are deficient when it comes to mitigating the risks associated with maintaining and improving systems. 

    Modernization: A Knowledge Problem

    With 10,000 mainframes in active use globally, the average organization annually spends anywhere from 60% to 80% of its IT budget just on maintenance. But, as experienced and proficient programmers retire or move on, many companies are realizing that the specialized domain knowledge of the industry and institution that those developers used to create and update these complex critical systems is walking out the door with them. 

    As the developers who originally wrote the code have long since moved on, and documentation leaves a lot to be desired, developers new to a system are unable to acquire comprehensive knowledge of the system’s functionality, at a detailed or fast enough level, to become adequately productive and proficient. That lack of speed comes with significant risk and cost to any organization.

    Recognizing the full and true intent of the functionality written in code is not simple. Developers today spend roughly 75% of their time searching through source code, to identify the code representing the functionality that needs to be changed. Complicating matters further, understanding the code is not enough. To effectively and efficiently maintain and support any system, software developers must know precisely what an application actually does—and how altering code in one part of the system impacts it as a whole. But since the code representing a behavior that needs to be changed can be littered throughout the system, developers might think they’re making a simple tweak, when really they may be breaking the entire system (and not even knowing it). The risk of change resulting in unintended consequences is real.

    All of that being said, how can developers improve their efficiency and effectiveness?

    The Shortcomings of Today’s Tools

    With code repositories growing unchecked over time to extraordinary sizes, those responsible for maintaining and preserving a system’s functionality say it’s becoming increasingly challenging to find bugs in code without machine assistance. And with debugging being an arduous, time-intensive endeavor, developers are increasingly turning to code search tools, linters, and static and dynamic analysis tools to analyze millions of lines of code, flag errors and suggest fixes. 

    Whether the tool excels at bug localization, code visualization or “error” detection, entirely too many tools are insufficient when it comes to actually identify the specific lines of code that require attention. Program analysis tools only illustrate code in ways that developers still have to interpret (potentially incorrectly) and draw their own conclusions. While code search tools might speed the rate at which developers can build a mental model of the code, such tools are notorious for false positives, and developers still have to undergo the mentally challenging endeavor of piecing that mental model together to identify the code, and then safely make changes. Worse yet, code completion tools—tools that less-experienced developers disproportionately rely on—can actually propose imprecise changes (one prominent tool offers incorrect code 71% of the time), and that introduces risk.

    The cherry on top of this problematic sundae is that today’s tools don’t have any way to effectively validate the scope and accuracy of a proposed change. The influence of a suggested change can be tough to identify, so testing for comprehensive validation is frequently next to impossible. 

    Ideally, there has to be a better way to mitigate risk while making changes to legacy systems. Enter artificial intelligence. By employing a unique approach to AI, the code repository truly becomes a knowledge repository, enabling many capabilities unrealized before now. 

    A Developer’s AI-Enabled Accessible Knowledge Repository 

    By leveraging AI to automate the identification of the specific lines of code that require attention, developers can simply ask this AI-driven knowledge repository where behaviors are coming from—and quickly identify the code associated with that behavior. This puts AI squarely in the position of intelligence augmentation, which is key to leveraging its capabilities. 

    This novel approach of AI reinterprets what the computation represents and converts it into concepts, therefore “thinking” about the code in the same way humans do. The result is that software developers no longer have to unearth the intent of previous developers encoded in the software to find potential bugs. Even better, developers are able to overcome the inadequacies of automated testing by using AI to validate that they haven’t broken the system before they compile or check in the code. The AI will forward simulate the change and determine whether it’s isolated to the behavior under change. The result is the bounds of the change are confined to the behavior under change so that no unintended consequences arise.

    While AI is not yet advanced enough to safely make changes on its own, it can serve as a knowledge repository with the added capability of directing developers precisely where to make any needed changes. With AI, programmers—even on their very first day on the job — can make any necessary fixes with confidence that they won’t break the entire system. 

    Unlike its code search, code navigation, code completion and program analysis tool counterparts, AI fundamentally empowers developers to safely, efficiently and effectively make any necessary changes. With AI — that is fortified by a knowledge repository that understands source code the same way a human does — developers can truly do more with less. 

  • NPMs Sabotaged as OSS Sustainability Crisis Continues

    NPMs Sabotaged as OSS Sustainability Crisis Continues

    A long-simmering debate over the sustainability of smaller open source projects moved beyond the theoretical when two widely used open source node packaged modules (npm) were deliberately sabotaged this week, allegedly by a primary contributor.

    Colors.js is an npm that has been downloaded more than 3.3 billion times, with more than 19,000 projects depending on it. Faker, meanwhile, has been retrieved 272 million times, with over 2,500 projects depending on it. Colors.js enables organizations to print colorful text messages on the console, while faker is used to generate fake data for testing applications. Developers that pulled a recently published Colors.js version found their applications caught in an infinite loop, printing ‘LIBERTY’ ‘LIBERTY’ ‘LIBERTY’ followed by a sequence of gibberish non-ASCII characters. Functional code, meanwhile, was also removed from Faker.

    Ax Sharma, senior security researcher for Sonatype, a provider of an open source software security platform, noted these actions occurred in the wake of the series of zero-day vulnerabilities that impacted the widely used Log4j logging tool for Java applications. Sharma first reported the update issues with Color.js and Faker.

    The small team of contributors that work on the Log4j project found themselves creating multiple updates to the package to address vulnerabilities of varying severity on the common vulnerabilities and exposures (CVE) list. There is some debate, however, over how many of those vulnerabilities warranted a CVE listing given their severity, said Sharma.

    That issue has now emerged as a flashpoint for contributors to smaller open source projects. These contributors contend that larger organizations are taking advantage of their efforts without making any substantial contributions to a project in return, much less compensating any of the contributors for their time and effort. The recent updates made to the npms are, essentially, a protest statement, explained Sharma.

    It’s not clear whether other contributors to small open source projects might follow suit, but the debate over the sustainability of these projects has become heated. Many contributors to open source software assume that the organizations who use the free software they created should assume responsibility for securing it. That “user beware” approach to security is understandable for contributors that are not compensated for their efforts. However, when asked to patch open source projects used by billion-dollar organizations—and do so on an urgent, emergency basis—the resentment among those volunteer contributors rises sharply.

    Fortunately, some effort is being made to address these issues. The Open Source Security Foundation (OpenSSF), an arm of the Linux Foundation, raised $10 million to help maintainers embrace best practices and better protect open source projects from malicious code. Google has pledged $1 million to help open source developers adhere to National Institute of Standards and Technology (NIST) guidelines in response to the Biden administration’s recent executive order on cybersecurity. Administered as a pilot program by the Linux Foundation, that effort is part of a larger $10 billion commitment that Google previously made to open source security.

    White House national security adviser Jake Sullivan also recently sent a letter to major software companies and developers inviting them to discuss initiatives to improve open-source software security. The first step is a one-day discussion this month hosted by Anne Neuberger, the deputy national security advisor for cyber and emerging technology. In the letter, Sullivan specifically noted that while open source software has accelerated the pace of innovation, much of it is maintained by volunteers. This is now a key national security concern, he noted.

    It’s not clear how this emerging open source sustainability crisis will play out. However, DevOps teams might want to consider how dependent they are on open source projects whose contributors might have issues with how their work is being exploited.

  • How Low-Code Enables the Composable Enterprise

    How Low-Code Enables the Composable Enterprise

    Enterprises know that their customers and partners expect a superior personalized experience. After all, customer experience has been a top priority for organizations over the last decade. However, despite all the technological advancements and transformation initiatives, the ability to deliver innovation with speed continues to elude. 

    Without this ability, enterprises constantly play catch-up with rapidly shifting business dynamics and customer expectations.

    Why is Strategic Agility so Elusive?

    Enterprises are unfortunately locked in a situation that Gartner Research identifies as “static application experience”. It is a paradigm of rigid and bloated application architecture, creating inertia that organizations find difficult to navigate around. This technological impedance is one of the primary reasons (besides organizational culture and structural challenges) at the root of organizational sluggishness.

    To adapt to the challenges of the modern world, enterprises need to move from a static application experience to a more dynamic one. This can be achieved through the approach — wrapped around a goal — of a composable enterprise.

    Composable Enterprise is the Future of Applications

    A composable enterprise aims to create an application architecture wherein enterprises can deliver various functions through composition (as against development), by leveraging packaged business capabilities (PBCs). Gartner estimates that by 2023, 30% of new applications will be delivered, priced, and consumed as libraries of packaged business capabilities, up from fewer than 5% in 2020.

    To be fair, this run-up to a composable enterprise is not a fresh-of-the-press revelation. Enterprises have been attempting to move from hardcore coding-based development to a more service-and-composition-oriented architecture over the last couple of decades, albeit only in pockets and not as fast as they would have wished.

    Composable enterprise has become the need of the hour. And this is being driven by the sense of urgency created by multi-faceted disruption across industries, coupled with technological advancements that make it possible for organizations to accomplish it at an enterprise scale. 

    One such advancement comes in the form of low code digital transformation platforms.

    How do Low-code Platforms drive Composable Enterprise?

    According to Forrester Research, low-code development platforms are emerging as a key strategy to accelerate app delivery to support digital business transformation. They have the potential to make software development as much as ten times faster than traditional methods. Low-code platforms use visual composition and a modeling-based approach to speed up application development. 

    A good low-code platform drives the creation of a composable enterprise in two ways, both of which are critical for long term sustainability:

    • By enabling rapid development of modern PBCs that deliver cutting-edge functionality without having to rebuild the organizational core, thereby protecting decades of investments in legacy and core applications 
    • By providing the core capability to create applications through composition and abstraction with a modeling-driven application environment, wherein the organization can create customer-centric business applications that deliver a dynamic omnichannel experience and drive the end-to-end customer journey

    Is There a Catch?

    Low code platforms come in various shapes and sizes, from those supporting the creation of relatively simpler stand-alone applications to those that enable the rapid development of mission-critical enterprise-scale applications with various functional components. Mostly, the latter category forms the core of a composable enterprise.

    To drive a composable enterprise, a low code platform needs to be equipped with a variety of capabilities. It should: 

    • Speed up pro-grade development through modular, modeling-driven, and abstraction-oriented development
    • Support a variety of business and technological needs, such as workflows, content management, straight-through transactions, mobile apps, collaboration, customer self-service, omnichannel customer engagement and automated business rules
    • Offer service-oriented integration that can extend beyond the organizational boundaries
    • Enable rationalization of application portfolio by helping avoid the introduction of multiple point-solutions in the mix with its own ability to develop a variety of applications quickly
    • Prevent shadow IT through central governance, visibility, and guardrails, without hindering innovation
    • Enable agile, continuous integration/continuous delivery (CI/CD) and adherence to quality standards and architectural requirements
    • Enable, even catalyze, change with traceability, thereby ensuring long term sustainability of business applications

    What Next?

    The ability of organizations to accomplish a composable enterprise of the future will depend on a multitude of factors. It is essentially a transformational journey.

    It is important to take the right steps and build in phases, starting from establishing a core low code platform and building peripheral capabilities with time. Picking the right set of applications and establishing early wins go a long way in building momentum towards the long-term goals.

    Identification of the right low code platform that can drive your transformation journey to a composable enterprise is critical for achieving a composable enterprise.

  • Step Aside, Monoliths: Meet the Composable Enterprise

    Step Aside, Monoliths: Meet the Composable Enterprise

    A composable enterprise, defined by Gartner as “an organization that delivers business outcomes and adapts to the pace of business change”, relies on the assembly of interchangeable application building blocks. This architectural overhaul has largely been driven by a demand for more configurable application experiences, and the need to evolve existing application portfolios that are often too risky and costly to replace.

    New business opportunities require agility from application portfolios; however, many enterprises are still limited in their ability to adapt. Why? They rely on monolithic ERP systems and cumbersome legacy applications with static processes and haphazard structures. A modular setup can enable a business to rearrange as required depending on external or internal factors, such as shifts in consumer attitudes or sudden supply chain disruptions. Organizations are experiencing these shifts now and require a new approach to enterprise applications to be able to adapt. 

    Bridge the CX Gap with Greater Composability

    A composability approach is the best way to capture all the advantages of modern enterprise software. According to a recent Boomi report, by 2023 organizations that have adopted a composable approach will outpace the competition by 80% in the speed of new feature implementation. This shift requires enterprises to rethink how they architect their operations, harness a combination of packaged functions and technologies and successfully deliver seamless moments of service to their customers.

    There are three key ways that businesses can make the composability shift:

    • Adopting a service mindset
    • Scaling the delivery of microservices
    • Packaging business capabilities using Application Programming Interfaces (APIs). 

    1. Designing for Service: Compete on Outcomes, not on Products 

    A composable enterprise whose portfolio is made up of heterogeneous applications from a palette of best-of-breed solutions allows organizations to address key inflection points throughout every customer, product or service lifecycle.

    As more consumers demand continuous value and reliability throughout an asset’s lifetime, businesses must shift to selling outcomes and experiences instead of products to meet these new expectations for quality service. This requires each part of an operation to align, not around immediate sales or revenue, but around delivering a quality Moment of Service™ — the inflection point where everything comes together to create better value and outcomes for customers. 

    Moving to a servitized model requires a composable stack to deliver services to order. At a systems level, this transformation requires organizations to adapt applications dynamically and deliver positive customer experiences with effective quality management, customer support, and access to complete information about the service offering. Unlike traditional enterprise software, this involves connecting data and applications that have often sat in separate silos. A composable enterprise can provide a service-orientated architecture, that enables businesses to become outcome-based and ready to quickly adapt to future disruptions. 

    2. Scale Component-based Architecture with Microservices

    If applications are built as loosely coupled services, then companies can employ a composable architecture to capitalize on independently deployable modules that are organized around business capabilities. This allows organizations to swap modules in and out, to suit emergent needs and build a well-structured best-fit solution for their unique business. 

    In contrast to a monolithic architecture, businesses can easily add resources to the most needed microservice rather than having to scale the entire application as demand for an application increases. In practice, this allows businesses to simplify customizable workflows and optimize business processes while leveraging and applying tools such as robotic process automation, artificial intelligence, or the plethora of hyper-automation capabilities available today.

    3. Embrace Open-ended APIs to Maximize Data-sharing Capabilities

    Packaged business capabilities (PBC) assembled using APIs are the foundation of every composable enterprise. They are used by businesses to secure data across cloud services, business systems and mobile applications. Historically, APIs have been used in monolithic applications to exchange data between the entire application and external applications and services. In composable software, APIs exchange data from individual modules to external applications and within the application — from module to module. This has significant implications for how systems are designed and built.

    APIs can provide a controlled and consumable method for connecting and sharing consumer and business data by creating experiences tailored to individual needs. For instance, an API-centric model can secure and manage data access to help businesses with faster decision-making and deliver relevant new services adaptable to market changes. In the future, there will be more automated continuous process improvements as machine learning models recommend, or even proactively make, process changes to improve outcomes for the business and end customer.

    Opt-in to the Composability Evolution

    To overcome the limitations of monolithic applications, businesses must rethink their approach to enterprise applications — starting with the business architecture and technology stack. A composable software architecture enables organizations to address the internal and external pressures that send shockwaves throughout the value chain. 

    As a composable enterprise, organizations can re-engineer their businesses to ensure customer touchpoints and stages come together for better moments of service, but companies must be certain that processes are optimized across each of these inflection points to mitigate issues and fuel growth. An IT architecture built on a foundation of composability will be essential to the successful delivery of a software-powered business development strategy that can provide continual value to customers and the business itself. 

  • GitLab Revamps Observability Strategy After Acquiring Opstrace

    GitLab Revamps Observability Strategy After Acquiring Opstrace

    GitLab plans to make observability a more integrated element of its continuous integration/continuous delivery (CI/CD) platform after acquiring Opstrace.

    Opstrace’s open source project exposes a horizontally scalable application programming interface (API) created by Cortex for Prometheus, an open source monitoring platform being advanced under the auspices of the Cloud Native Computing Foundation (CNCF). It then makes use of a Loki API to collect log data.

    DevOps teams can also point an existing instance of Prometheus to Opstrace or any instance of Fluentd or Promtail software they are using to collect log data, with support for a wide variety of other APIs, such as the Datadog agent, planned. A command line interface (CLI) is provided to set an instance of Opstrace that creates a controller through which the collecting of log data is managed.

    Kenny Johnston, senior director for product management for GitLab, said as part of an ongoing effort to make observability more accessible, GitLab plans to integrate Opstrace by default with GitLab Monitor for organizations that use either the self-managed version of GitLab or the software-as-a-service (SaaS) platform.

    Opstrace currently collects logs and metrics and support for distributed traces is planned. The goal is to extend the overall scope of observability capabilities that GitLab already provides within its CI/CD platform, noted Johnston. The goal is to make it simpler for DevOps teams to employ observability capabilities and interrogate data to optimize workflows without acquiring a separate observability platform, he added.

    GitLab is betting that, as Prometheus continues to gain traction, more DevOps teams will prefer to extend its reach into the realm of observability rather than having to manage a separate platform.

    Although observability has always been a core tenet of DevOps best practices, achieving it has proven to be a challenge. At best, most organizations can continuously monitor IT environments based on predefined metrics that identify when a specific platform or application is performing within expectations. Observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshot issues without having to rely solely on a limited set of pre-defined metrics that have been created to monitor a specific process or function.

    The rate at which DevOps teams will embrace observability will naturally vary. In 2022, the biggest obstacle might not be the technology as much as it is simply understanding what queries can help DevOps teams better understand the root cause of an IT issue before there is a major disruption. In the long term, it’s expected that machine learning algorithms will leverage the data collected by observability platforms to automatically identify issues that might lead to a disruption long before it actually occurs.

    One way or another, the ability to discern events even within the most complex IT environment is steadily improving. Whether that’s accomplished as an extension of monitoring or by embracing true observability may not be all that relevant—as long as the ultimate goal is achieved.