devopsy.kinsta.cloud

Category: DevOps and Open Technologies

Editorial Calendar for February 2016

  • 4 Things to Consider When Integrating Generative AI

    4 Things to Consider When Integrating Generative AI

    Generative AI has emerged as one of the most influential innovations of the past two decades. ChatGPT gained 100 million users in record time, and new generative AI-powered point solutions emerge every day.

    Considering the current state of VC funding, the surge of investment dollars in AI companies is remarkable. Microsoft’s $13 billion investment in OpenAI and AWS’ $4 billion investment in Anthropic (who themselves just raised a $850 million Series C round) demonstrate the tremendous excitement around Generative AI and how it will shape the future of tech.

    As newer companies like Adept and Cohere are emerging, existing SaaS companies are integrating generative AI capabilities into their products and services. However, in the race to join the hype, many companies are building first and strategizing later. Before diving headfirst into building a generative AI solution, companies should carefully consider a few crucial factors, including model selection, usage tracking, monetization and customer transparency.

    1. AI Model Selection: To Build or to Buy?

    Companies must first consider whether to build a generative AI model from scratch, or whether to use a pre-existing model from one of the major AI vendors. This decision depends on the level of customization and control desired, as well as the resources and expertise available.

    Building a from-scratch model is likely infeasible due to the inaccessibility of the necessary hardware. The GPUs needed for cutting-edge model training and inference are currently only built by NVIDIA, and there is stiff competition and long lead times for any company looking to acquire the product. What’s more, large cloud and AI companies like OpenAI, Google, Meta and Tesla already have built systems with up to 100,000 of these chips. New companies looking to enter the scene would face a steep climb to assemble the infrastructure needed to even begin to compete with the current classes of models.

    Small or new companies do not have the funding or time to invest in building their own infrastructure or generative AI models, so a key business decision is to select the best vendor and model for the use case. While ChatGPT is the most well-known, there are other specialized models that can be better suited for certain use cases. For example, OpenAI’s DALL-E is built for image generation, while Anthropic’s Claude 2 is built to perform better with specialized tasks such as legal and math and to handle long-form inputs.

    Regardless of the AI vendor you choose, you will be charged based on your usage with them, which, by extension, is your customers’ usage. Tracking this usage is critical to your generative AI solution becoming profitable.

    2. Metering AI Service Usage: The Key to Success

    Implementing usage metering for generative AI services is crucial at both the individual user level and at the global or system-wide level. This is because usage tracking is valuable to all business functions and is a critical element of any modern PLG strategy.

    For example, customer-facing teams can leverage usage data to proactively track account health and to contextually recommend new solutions and approaches. Usage history provides valuable intelligence to account teams for up-selling, cross-selling, and contract renewal. Product teams can identify usage patterns and customer segments, as well as areas of friction or improvement for new feature development.

    Usage data also plays a major role in developing pricing plans that align with customer expectations and overall market trends. It can be used to inform pricing plan development and optimization to ensure the rates are sensible for the level of consumption and value being realized. Without tracking usage, this becomes a guessing game or a futile exercise in “gut feel.”

    Usage data is imperative to calculating your internal costs. Vendors commonly charge according to a usage-based model for these generative services–per word, per token or per generation are some common models. In any case, having accurate, granular usage data is a must-have for calculating internal costs and, in turn, customer-facing pricing.

    3. Monetizing Generative AI Services: Usage-Based Pricing Strategy

    In the rush to beat competitors to market and offer generative AI services to customers, do not neglect to lay the groundwork for monetization from day zero. It is much easier and more efficient to build with monetization in mind from the start than it is to backtrack and try to monetize a service once it has been offered, particularly when usage-based pricing is involved.

    Usage-based pricing has emerged as the optimal pricing strategy in the AI domain. This is because service usage varies over time for each customer. These services are priced by the vendors according to a usage-based pricing model. Since the backend costs of the service are elastic, the frontend, customer-facing pricing should also be elastic.

    As a roadmap to monetization, follow these steps:

    -Meter service usage to understand segments and patterns

    -Calculate internal costs of usage using vendor pricing

    -Using the information from steps 1 and 2, build customer-facing pricing

    -Track revenue and margins and continue to iterate with pricing as needed

    4. Customer Transparency: On-Demand Access to Usage and Billing Data

    Prioritizing transparency and clear communication with customers is non-negotiable. Ensuring that usage and billing data can be seen at any point in the billing cycle helps foster trust between the company and its users and eliminates billing surprises when the invoice is delivered.

    Adopting Transparent Communication Builds Trust With Customers

    Invest in creating a customer-facing application, portal or experience that allows for on-demand access to usage and billing data. The beauty of combining this level of visibility with usage-based pricing is that customers are empowered to pay for only what they use while viewing usage statistics on demand and modulating usage as needed in real-time to keep the spend within an acceptable range.

    In practice, usage data is available on a per-query level from the vendor’s APIs. For example, for any generation action, OpenAI will show the precise number of tokens consumed for both the prompt and response, as well as all details around the model type and context needed to identify the per-token rate from the vendor pricing. It is important to collect this data and build a user-friendly way to display it back to customers in an app, portal, or website that allows for on-demand access and browsing.

    Conclusion

    While planning to build a product with generative AI, take all of the above factors into account. Do not make the mistake of racing to deliver a technology solution without also considering the business implications and monetization strategy. With all the hype, excitement and investment around generative AI in 2023, it is akin to a modern-day gold rush. Don’t show up to the gold rush without a shovel.

  • Multi-Tenant Database Strategies for Mixed Workloads and Real-Time SLAs

    Multi-Tenant Database Strategies for Mixed Workloads and Real-Time SLAs

    Multi-tenancy—where a single instance or multiple instances of one or more applications operate in a single shared infrastructure environment—is not a new concept. First used in mainframe hardware, timesharing enabled a single massive piece of computing iron to be sliced up and used by multiple applications on a single hardware instance. This simplified both the hardware landscape as well as the skills and resources needed to manage environments. With the rise of software-as-a-service (SaaS) in today’s modern private and public cloud environments, multi-tenancy is now a common and critical function. It allows cloud companies to scale and customers to save money by essentially pooling resources. As more applications demand more and more data, multi-tenancy is now a core function at the database layer.

    But when you have a lot of different applications running on the same infrastructure and accessing the same database, how do you prevent one application from impacting the data performance of another? And how do you determine from the underlying database infrastructure which tenants get what infrastructure and at what performance level?

    Here’s a look at the driving forces behind today’s real-time data needs and how they impact multi-tenant data architecture management.

    SLAs Are at the Heart of Multi-Tenant Data Management

    Everyone wants faster performance and more resilient systems. We’re impatient, so reliable, real-time performance delivers large, competitive business benefits. However, every tenant does not require the same level of performance. Actual performance needs may be plenty fast for a user or machine without requiring the fastest execution path possible with the available resources.

    Service-level agreements (SLAs) ultimately govern the resources delivered to a tenant. Real-time may need to be measured in sub-milliseconds in one instance (i.e., for a stock trade) but in sub-seconds for others (i.e., in a Zelle funds transfer). This is where developers, architects and operators partner with a business to understand its actual user or system needs and to build a system that can deliver the SLA using the minimum amount of resources possible.

    Mixed Workloads and the ‘Noisy Neighbor’ Problem

    As soon as workloads are shared, there’s the potential for one application to quickly grab and use resources in a way that impacts other applications. Often called a “noisy neighbor,” the impacting applications can dramatically interfere with real-time response time SLAs when multi-tenant apps share an infrastructure base. For example, one node in a database cluster becomes heavily loaded, and the performance of other nodes begins to be impacted, causing SLAs to be missed.

    Typically, multiple virtual machines (VMs) are hosted on a single physical machine or node but share system resources (i.e., processor, network, storage, etc.) with the other VMs. It’s possible for applications to overuse those shared resources, impacting other applications and sometimes even the database itself. Such overusage of the infrastructure layer (in this case, sharing compute, storage, network queues, etc.) leads to poor database and end-to-end system performance. It’s like plugging a 100-amp appliance into a 50-amp circuit. All the circuit breakers will trip, and the entire system comes to a screeching halt!

    Strategies to Maximize Performance for Database Multi-Tenancy

    Configuring and managing the database itself (not just infrastructure) with multi-tenancy in mind can maximize performance and avoid problems like noisy neighbors. Isolating databases in your multi-tenant architecture can help minimize non-database problems that can be the root cause of poor performance and missed SLAs.

    Start by picking databases that allow applications to isolate their data access based on sets (or tables in relational parlance) that can have storage limits attached to them. When combined with rate limits for transactions on a per-user basis, applications can be guaranteed their quota of runtime resources. Using the per-app quota configurations for items like storage and TPS (transactions per second), the database can evaluate the total capacity available in the system and ensure that enough resources are reserved and available to ensure that the SLAs of all applications are met all the time.

    Note that unexpected events that cause rebalancing of data within the distributed database, such as node and network failures, require additional resources. The database system will subtract that resource capacity from what is available for apps so that the system is not overcommitted during failure situations.

    Enterprises also need very good automation and observability to fix immediate problems quickly and have the requisite visibility into long-term trends that impact performance and SLA compliance. Otherwise, by the time a human notices a problem, the SLA may already have been missed and the business impacted.

    Good observability can help you understand how data is used and create those automations. As more applications access and use the same data set in different ways, you need modern observability tools integrated into (or interoperable with) the underlying database.

    With a good understanding of how your data-rich applications interoperate with other tenants in mixed-load environments, you can begin to deploy these strategies to make sure you conform to all required SLAs and keep all that data working hard for the business.

  • Microsoft kills Python 3.7 ¦ … and VBScript ¦ Exascaling ARM on Jupiter

    Microsoft kills Python 3.7 ¦ … and VBScript ¦ Exascaling ARM on Jupiter

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: VS Code drops support for Python 3.7, Windows drops VBScript, and Europe plans the fastest ARM supercomputer. (more…)

  • How Event-Driven Architectures Drive Real-Time Operations

    How Event-Driven Architectures Drive Real-Time Operations

    People, events, the human brain—in fact, the whole world operate in real-time, but businesses have struggled to keep up. With the help of event-driven architecture (EDA) and the Open API economy, businesses can now do the same. The power of an event-driven world means that, after years of geopolitical events affecting how businesses operate, many businesses are starting to uncover real value by truly being able to operate in real-time.

    Whether it be retail and manufacturing or energy and resources and financial services, locating and responding to vital issues within a company’s supply chains or product lines in real-time, is key to success.

    Amazon’s CTO, Dr. Werner Vogels, said that “the world is event driven” in his keynote speech at AWS re:Invent in December 2022. Now, new IDC research unveils that nine out of 10 of the world’s largest companies will deploy real-time intelligence driven by event-streaming technologies by 2025.

    But What’s the Secret Behind Such Success?

    A recent IDC Infobrief, sponsored by Solace, surveyed over 300 enterprise IT professionals in North America, Asia and Europe, all of whom work for large companies implementing or considering EDA. The results are quite telling–an overwhelming 93% of respondents at companies that have deployed EDA across multiple use cases said EDA has either met or exceeded their expectations. In addition to technical advantages from EDA, most businesses also see clear business benefits: A full 23% of respondents reported increasing productivity; 22% said better customer acquisition and 18% saw revenues increase as a result of EDA efforts.

    1. Get Support From the Top to Ensure Alignment Throughout

    Expanding the footprint of EDA across the enterprise is a journey, and every journey starts by assembling those that are critical to its overall success. Business sponsorship and engaging key stakeholders is vital, especially in the early days of EDA adoption – 56% of respondents in the early EDA stages cited this as a priority when ROI and business benefits may not be immediately clear.

    The impact of well-aligned C-suite, operational and technical teams is reflective of business-level digital maturity, too. As 35% of respondents at an advanced stage of EDA rollout felt C-level support was critical, it comes as no surprise that respondents with higher levels of EDA maturity also have higher levels of overall digital maturity, including digital strategy and change management support.

    2. Tackle Complexities Head-On With the Backing of IT

    As EDA becomes more pervasive across an organization, demands on IT become more sophisticated, requiring a deepening of EDA skills in the IT organization, notably with DevOps teams, developers and architects. More than one-third (36.1%) of respondents cited the lack of skills to execute EDA as a hurdle to adoption. Approaches to logging, governance and oversight (30.7%) can also become increasingly challenging and must be thought through carefully.

    This is where EDA providers themselves need to step up and provide adequate training and a certification path for architects DevOps and developers looking to gain the fundamental knowledge and skills to design and implement event-driven systems. This should include technical details such as understanding various design patterns for EDA, microservices choreography versus orchestration, the saga pattern and RESTful microservices. Education should also clearly define and demonstrate key concepts and tools for EDA success, such as event portal, topic hierarchy best practices and event mesh.

    3. The Benchmark for Success Chops and Changes: Businesses Need to Go With the Flow

    The right measures of EDA success in the early days of adoption are not the same measures as EDA adoption grows over time.

    Fewer EDA-mature organizations cited “cost reduction” (23%) and “number of projects completed” (31%) as top measures of success. In the medium term, after two or three use cases, most organizations saw “operational stability” along with “number of projects” to measure success. Further, after broad adoption, “increased revenue” (43%), “operational stability” (32%) and “amount of resiliency” (30%) become more important measures.

    This changing set of measures is reflective of the maturity curve of EDA adoption across multiple applications within a business’s ecosystem. A business is truly operating in real-time when deploying EDA across key customer-facing, employee-facing and supplier-facing organizations. This is where revenue is being generated in high volumes, organization-wide – hence stability and resilience become key to ensuring things continue to run smoothly on what can often be a global scale.

    4. Keep Everyone in the Loop

    It is essential to take the company with you along the entire journey. The research showed the bumps in the roadmap. In the early stages of EDA maturity, lack of understanding of EDA benefits and inconsistent buy-in are the most frequently cited organizational challenges (38%). As organizations progress to a centralized EDA status, cost concerns (42%) and finding the right use cases (36%) can hamper success. More advanced stages of EDA also cite a lack of understanding of EDA benefits (45%), and cost concerns (39%) but change resistance (38%) becomes a major hurdle.

    Continuous focus on change management and communicating business benefits is vital. The core group of invested stakeholders who began the journey must band together to demonstrate the value of EDA clearly and consistently to other areas of the business on a regular basis.

    5. Recognize the Benefits That Partners Bring to the Table

    To accelerate business and technical benefits and reduce the impact of challenges, survey respondents cited “finding a supportive partner to assist with the implementation of EDA” (37.7%) as a critical consideration. Partnerships and product integrations with the providers of preferred business software and SaaS services can make it easy to event-enable an organization. Given the amount of new technical skills and learning previously cited, this can provide peace of mind for organizations taking their first steps on the journey to EDA.

    EDA isn’t just a change in IT architecture, it’s a mindset that applies to modern business leadership organization-wide. To keep the momentum during the journey, respondents also noted the use of a robust change management plan that “ensures coordinated middle management support” (35.1%), as they are typically accountable for getting work delivered and reporting across the organization.

    EDA Becomes the Norm

    The InfoBrief results speak for themselves – EDA is quickly becoming the “normal” process for businesses looking to operate in real-time. As more and more companies recognize the business benefits at stake, understanding these five requirements will be a necessity for successful EDA deployment.

  • Avoiding Cloud Lock-In Amidst Potential Regulatory Change

    Avoiding Cloud Lock-In Amidst Potential Regulatory Change

    At the end of my street, there’s a beautiful and much-admired garden with vibrant flowers and a well-manicured lawn. Next to this splendid garden is an abandoned vacant lot, long-neglected and overgrown with weeds. Every time I walk by, I think about how this neglected space could be transformed into something magnificent (like this garden), but no one wants to take responsibility for it. Similarly, many CIOs and CTOs have “discarded vacant lots” to deal with in their own work lives: The problem of cloud lock-in.

    Cloud lock-in can be a significant headache for business leaders. Even the UK’s communications sector regulator, Ofcom, has expressed concerns about the issue. Earlier this year, Ofcom announced that it would refer the UK cloud market for investigation to the Competition and Markets Authority, highlighting barriers to interoperability and concerns about high egress fees charged by the cloud hyperscalers.

    But organizations shouldn’t just wait (and hope) for regulators to solve this problem. As a true believer in harnessing the full benefits of the cloud, I propose two practical ways to combat cloud lock-in.

    The Upside of a Technical Approach

    The first option is straightforward: Have an honest conversation with your cloud service account leader. Express your concerns about lock-in and negotiate more favorable terms and conditions. This approach might yield some results, but it’s not a guaranteed success. So, there’s a more effective strategy: Adopt a technical approach that empowers you to avoid vendor lock-in altogether, without costing you an arm and a leg.

    Imagine that, instead of trying to turn your neglected space into a garden alone, you could access a ready-made gardening community, where multiple people work together to create something amazing. The same principle applies to cloud databases. Rather than relying exclusively on one hyperscaler’s technology stack, you can opt for platforms that offer portability and flexibility across different cloud providers.

    Let’s consider databases, a critical underlying component of many mission-critical business applications. Each cloud hyperscaler will offer a range of SQL, NoSQL, and graph database options, which cater to various types and scales of technical problems. The challenge is to avoid getting locked into a single cloud stack, which hinders your ability to switch between clouds if and when needed.

    One viable solution is to choose “vanilla Postgres” – the basic open source version of PostgreSQL available from postgres.org or any cloud provider. This will certainly provide initial flexibility, but you are likely to encounter limitations as your application grows and requires enterprise-level features.

    The real benefit of the cloud lies in modernizing your data layer to fully exploit cloud-native features like scalability and resilience.

    To achieve this, you need databases that run across different public clouds and share common APIs. Born-in-the-cloud technologies, designed from the ground up for cloud environments, offer the most flexibility and benefits.

    Examining your current cloud vendor’s offerings is natural, as you’re already on their platform. However, keep in mind that these services are customized to their specific cloud environment only, so this could limit your future growth. If you decide to change cloud providers or adopt a multi-cloud strategy, moving across the cloud database “matrix” can be cumbersome due to the inconsistent APIs.

    If you were restricted to a single set of gardening tools when trying to transform that discarded space into a beautiful community garden, it would be nearly impossible and would not be fit for purpose.

    You need to realize that these services and their functionality will always need to be customized in some way to the specific cloud environment. AWS’s fully managed relational database engine, Aurora, is compatible with MySQL and PostgreSQL, but it was specifically designed to run on Amazon. This means it can’t run on-prem, and it certainly won’t work in GCP (Google’s cloud).

    As a consequence, you will have to make time-consuming and costly changes to your application to accommodate a bespoke version of PostgreSQL that will run efficiently in the cloud environment of your hyperscaler partner. You’re going to be locked into that version of the application because it can’t really work anywhere else. The same applies to the Google equivalent of vanilla Postgres (CloudSQL for Postgres), or the Microsoft version (Azure PostgreSQL).

    If you truly want a multi-cloud strategy, you need to consider how you go, for example, from mid-range Google to low-end Amazon. Scaling is much discussed, but surely one of the main selling points of cloud is that you can go down as easily as you go up.

    The Business Advantages of Multi-Cloud are in Reach

    To achieve the advantages of multi-cloud, you need to consider databases that offer common APIs and portability across clouds. There are several open source PostgreSQL-based database options available, which enable you to keep your cloud database options open and avoid the trap of cloud vendor lock-in.

    The wait for regulatory intervention might be long and uncertain. Taking proactive steps now to mitigate cloud lock-in is crucial.

    Adopt a scalable open source PostgreSQL option to immediately enjoy many of the benefits of multi-cloud. Embracing this strategy now can spare you potential panic moves when and if regulatory changes come into play.

    By freeing your business from the shackles of cloud lock-in, you can unleash the true potential of multi-cloud environments. Just like that “discarded vacant lot” waiting to be transformed into a beautiful garden, your applications can blossom in a multi-cloud ecosystem, which allows you to easily scale up or down and enjoy the best that each cloud provider has to offer.

    So, don’t wait for someone else to fix the problem. Take action today and start cultivating your multi-cloud future.

  • Raspberry Pi 5: Faster, Better, Stronger — Spendier

    Raspberry Pi 5: Faster, Better, Stronger — Spendier

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    In a cheeky extra post this week: Everyone’s favorite single-board ARM computer, the Raspberry Pi, has a new generation coming soon. Compared to the ’4, RPi5 has double the performance, quadruple the base RAM and far more capable I/O. (more…)