devopsy.kinsta.cloud

Category: Blogs

  • JFrog Acquires Upswift to Push DevOps to the Edge

    JFrog Acquires Upswift to Push DevOps to the Edge

    JFrog today revealed it has acquired Upswift as part of an effort to extend its DevOps reach to edge computing platforms. Terms of the acquisition were not disclosed.

    Upswift has created agent software that is deployed as a binary file on an edge computing platform that makes it possible to continuously deliver software across a fleet of edge computing devices and platforms. That approach makes it possible for organizations to deploy software regardless of whether the target platform is only running Linux, containers or a virtual machine.

    JFrog CEO Shlomi Ben Haim said the agent software developed by Upswift will be integrated with the continuous integration/continuous delivery (CI/CD) platform created by JFrog to make it easier for organizations to manage the full life cycle of application workloads deployed on edge computing platforms.

    JFrog Upswift

    Organizations that deploy software on edge computing platforms and devices today typically have massive backlogs of updates that they are unable to deploy in a timely manner, said Ben Haim. By connecting the Upswift agent software to a (CI/CD) platform designed from the ground up to support binaries, it becomes possible to reduce that backlog as part of what JFrog describes as its ‘liquid vision’ for deploying software, he noted.

    In addition, after acquiring Vdoo to gain access to a set of analytics tools that discover vulnerabilities in application binaries, JFrog will also be able to bring DevSecOps best practices to edge computing applications as well, noted Ben Haim.

    The number of application workloads being deployed on edge computing platforms is rising sharply as organizations seek to process and analyze data closer to the point where it is being created and consumed. In fact, that shift is at the core of digital business transformation initiatives that depend on edge computing platforms and devices running, for example, internet of things (IoT) applications to process data in near-real-time.

    Upswift CEO Amit Ezer said it has already become apparent that the existing processes of updating software on edge computing platforms and devices are capable of handling that influx of software. Upswift created its tool to enable developers to assume more control over those processes, as responsibility for managing and securing software continues to shift left, he added. The issue in many cases is that organizations lack the ability to consistently deploy software on edge computing platforms, which results in a hodgepodge of homegrown tools being created to deploy software on edge computing platforms using largely manual processes.

    It’s too early to say just how much software will be deployed on the edge, but there may come a day when there is more software at the edge than there is in the cloud. Of course, the real challenge will be unifying the management of DevOps processes, regardless of where any given piece of software might be deployed. In fact, as edge computing continues to evolve, the line between edge computing and cloud computing will continue to blur until they are little more than natural extensions of each other.

  • 4 Benefits of Integrating a Language With a Database

    4 Benefits of Integrating a Language With a Database

    One of the hallmarks of a truly modern programming language is the coupling of a language—for creating applications, devising algorithms and solving business problems—with an underlying database. When there is practically no separation between a database and its programming language, developers immensely increase their productivity, maximize the effectiveness of the code they write and provide unparalleled speed and flexibility in supporting business needs.

    Predicating a programming language on a database delivers these advantages by enabling developers to treat data as if it is in-memory, liberating them from the burden of manipulating how data is represented in the underlying database. This one simple advantage exponentially increases the flexibility for addressing business problems and, when used with advanced logic languages, enables developers to write applications in a fraction of the time and with less effort than it would otherwise require.

    Specific benefits of tightly coupling a database with its programming language include eliminating the time-honored mismatch between data structures and database representations, smart caching with transactional technologies for committing or rolling back database changes and automatically changing object definitions.

    These benefits remove the database as an otherwise complicating factor in supporting business use of data by freeing developers to flexibly innovate solutions—and speedily implement them.

    In-Memory Manipulations

    Developers focus on manipulating data structures to solve business problems. Traditionally, this concern was hamstrung by the onus of translating those structures into how data are represented in the database, a time-consuming necessity detracting from programmers’ attention to business problems. By fully integrating an object-oriented database or graph database with its programming environment, users can manipulate data objects in memory and they’ll automatically persist to storage. Developers can solely focus on working with data structures for business objectives without spending just as much, if not more, effort wrangling data’s representation in the database.

    Programming languages create this effect when they are imbued with an intelligent caching layer that persists in-memory data manipulations into the database. This advantage requires basing the programming language on a specific database. It eliminates the mismatch between working on data structures and toiling to get the underlying data to and from disk.

    Smart Caching

    The caching layer also tames the inherent difficulties in data objects pointing to other objects, which is a hallmark of object-oriented systems and graph databases. These databases require intelligent caching because one object can conceivably include the majority of the database by pointing to objects that, in turn, point to additional objects. Tight coupling between that language and its database fixes this issue so developers cache what is needed to work on data in memory while automating its impact on disk. Such caching is significantly improved with transactional technologies so developers can specify when they want their modifications saved in the database.

    This way, if they make a mistake they can simply go back in time before it was made and undo it. Conversely, after successfully completing their work, they can easily commit changes to the database. Like the other advantages, this benefit boosts developer productivity to match that of business requirements. It adds to the flexibility programmers need to innovate solutions for mission-critical objectives by writing or integrating applications.

    Object and Class Definitions

    The third boon of basing a smart programming language on a database is the capacity to dynamically change the definition of data objects. With conventional approaches, this task requires developers to stop writing or operating applications and manually write code to adjust the classification, which may involve either reading or writing the data in and out again. But with no separation between the database and its sufficiently smart programming language, such coupling can effectively keep track of classifications while automatically modifying them as needed.

    Developers can seamlessly add or delete features without manual coding, which also furthers their productivity. For example, if they add an attribute to an object, the next time they see it it’ll magically have the new feature. Alternatively, if they create a new feature of an old classification and decide to delete it, they won’t see it anymore. Best of all, these changes occur without rewriting the classification.

    Advanced Logic

    There is one more capability of very smart programming languages: They can combine these advantages with advanced logic languages such as Prolog. Prolog, or a subset of Prolog called Datalog, can instantly be used as a query language. In addition, it becomes very easy to create rule-based applications with Prolog because one can write declarative, logic-based rules with minimal coding. For example, they can specify a series of event-based rules detecting customer churn, money laundering or any important business outcome without being impeded by unnecessary database concerns to deliver time-to-value that’s unmatched without this method.

    Basing a programming language on a database improves developer efficiency and effectiveness by letting developers work on data in memory and persisting changes to storage, intelligently caching data and rolling back changes as required, and dynamically adjusting definitions.

  • Avoiding Budget Overruns During Cloud Migration

    Avoiding Budget Overruns During Cloud Migration

    Many organizations have already started their migrations to the cloud, or soon plan to. It makes sense: The cloud is an attractive option for multiple reasons, including its potential for cost savings, scalability and support of continuous application development.

    However, benefits aside, it can be very easy for enterprises to lose financial control of a cloud migration. The potential for costly mistakes when shifting to the cloud is significant, and organizations often end up getting hit with surprising bills that ultimately stop them in their tracks. In fact, in a 2020 survey, 94% of participants claim that it is a challenge for them to take full advantage of public cloud resources, with 40% citing the need to control cloud costs as their biggest obstacle.

    These financial mistakes are typically made due to the pay-per-use model of cloud services and the growing cloud skills gap. Too often, IT team members are forced to work in cloud environments they are not proficient in, and as new features are released every week, it’s almost impossible for these individuals to keep up. One recent study claimed that 90% of enterprises found that their cloud skills gap doubled over a three-year period.

    The Two Different Types of Cloud Migrations

    There are two types of cloud migrations that typically occur, a lift and shift, or a replatform/rearchitecture migration. The lift and shift method is fast and relatively easy, but it doesn’t enable organizations to truly optimize their cloud usage. The rearchitecture method is the most popular as it gives an organization the opportunity to fix any outstanding issues by harnessing the benefits of cloud-native technologies and services.

    However, once a rearchitecting migration begins, organizations often find it takes significantly more time, effort and resources to complete than they originally thought. This can become a real challenge, especially when facing critical deadlines or experiencing major events like the COVID-19 pandemic or the wild weather events experienced around the globe.

    Understanding and Assessing Performance

    For both types of cloud migrations, understanding and assessing performance and underlying architecture is an ongoing task, as cloud providers are always releasing new or improved services or lowering prices. To make sure they’re achieving the best performance and building a resilient architecture, application developers and IT teams must always be looking for ways to improve their applications. Within the cloud, this is typically focused on four key areas: Networking, computing, storage and application architecture.

    From a network perspective, understanding where data is being used and where data is being accessed is critical and can help organizations cut costs exponentially. For example, cloud providers often have costly data ingress and egress charges, so assessing how best to access and use data when necessary can help a company navigate those charges.

    Regarding computing, there are multiple resources that can help manage system load, make tweaks to computing infrastructure and find performance and cost benefits. For instance, leveraging the data and metrics from native cloud reporting tools can help organizations understand and predict demand, and taking advantage of services like auto-scaling enables them to scale up or down appropriately depending on the changes in application use and demand.

    On the storage front, cloud providers offer a number of ways to save money and boost performance. From the release of GP3 volumes to tiered long-term storage options based on the frequency of access, storage is an easy way to find cloud savings opportunities. Additionally, cloud providers will usually make recommendations and set up tiering policies to help move data automatically into colder and deeper storage.

    As cloud providers continue adding new services, looking at your application architecture is essential to innovation and cost savings—especially when including labor and effort—and truly achieving the promise of the cloud.

    Solving Common Costly Mistakes

    While there are numerous examples of common financial mistakes companies make when managing cloud instances, lack of visibility and understanding are the leading pain points for the developers who stand up and maintain their organization’s architecture. With the demand for cloud so high and with their attention directed elsewhere, developers and engineers often take the mentality of “If it isn’t broke, don’t fix it.” This hampers the company’s ability to assess and optimize their cloud operations. They don’t have time to consistently review their applications for ways to save money or even know how much their decisions cost.

    This ubiquitous issue can be solved by bringing greater visibility into the cloud migration and management process and aligning real-time finances directly with established corporate budgets. In addition, organizations struggling with the cloud services’ pay-per-use model should take the necessary steps—whether internally or with a third-party vendor—to ensure cloud resource and service use is being effectively monitored. This makes it easier for developers to understand where and how their actions affect the financial posture of the organization before their boss comes knocking on the door.

    If enterprises provide the resources necessary for IT teams to become more knowledgeable about how their applications are being used within the cloud and address the cloud skills gap in their workforce, they will be able to prevent cost overruns, experience cost savings and take back control of their budgets. This can take a variety of forms, but enterprises need to establish continual training programs to ensure team members are up-to-date on the latest cloud features and services, as well as leverage tools and automation to reduce manual intervention. Providing this critical visibility throughout a cloud migration can often mean the difference between a successful, on-budget migration and one that is stalled and costly.

  • Power and Influence

    Power and Influence

  • 3 Pillars of Autonomous IT Ops

    3 Pillars of Autonomous IT Ops

    Imagine a future where human IT operators and intelligent systems work together in a virtual operations room; the systems do the heavy lifting while the humans supervise. This may seem like science fiction, but the truth is we are already on the way to that future with AIOps. While actual AI-augmented teams are still quite a way off, AI/ML tools are slowly but surely becoming more ubiquitous within IT Ops, NOC, DevOps and SRE teams and leading us into this not-too-distant future.

    In the last five years, the number of internet users grew by 50%. In the last two years alone, more data was generated than in the entirety of human history up to that point. Today, 5 quintillion (that’s 18 zeros) bytes of data are produced by smart devices daily. These massive volumes of data are having a huge impact on digital operations.

    Looking at the enterprise, we see similar trends: More tools generating more data and with it, greater complexity. This is due to many factors, including the transition to the cloud, M&A activity that introduces new tools and data into an existing organization and the adoption of new technologies and methodologies such as CI/CD, microservices and more. At some point, there is so much data and everything becomes so complex that IT Ops teams reach a breaking point where they can no longer manually manage the data being generated.

    And when that point is reached, outages and service disruption become an everyday reality, as IT teams are pulled away from their strategic, business-critical projects into what is essentially firefighting. 

    Implementing more observability and monitoring tools simply exacerbates the problem, and scaling teams isn’t cost-effective in the long term. The more forward-thinking solution is automation, and even more so, autonomy. 

    The Road to IT Ops Autonomy 

    Autonomy is the ability of IT Ops to operate with minimal to no human intervention. It relies on the automation of the incident management life cycle so that humans are needed only for supervision, in the same way that a single operator can supervise a modern and highly complex robotic car assembly line.

    How can we hope to reach IT Ops autonomy?  The following three proposed pillars can act as a guide: 

    Pillar One: Automate

    Pillar Two: Predict and prevent 

    Pillar Three: Democratize

     Pillar One: Automate

    We must automate as much as possible across every step in every aspect of incident management. It won’t happen overnight, but with time and with the help of AI/ML it is doable.

    Consider the autonomous driving analogy.

    Just a few years ago, cars were manual, with little to no automation. Then we moved through partial autonomy to where we are today–conditional autonomy–where the car can drive itself in ideal conditions. This is a level of automation that we are beginning to accept; some of us cautiously, others with more enthusiasm! Meanwhile, progress continues toward higher levels of autonomy, in which vehicles will be able to drive themselves in most conditions. One day, maybe, we’ll get to the holy grail of full autonomy, where the car will drive on its own–perhaps we won’t even need a steering wheel!

     

    Similarly, in IT Ops, automation is a process and it will take time to build not only the capabilities required but also the necessary trust in AI/ML technology. Such a journey typically begins with human-driven automation, as “if-then-else” types of automation are added manually. For example, if a certain alert appears, a ticket will automatically be assigned to a specific person. As our confidence grows, the next stage could be AI-driven suggestions, whereby AI applies what it has learned from previous human actions to suggest what to automate, but the human expert decides whether to accept the suggestion or not. 

    At some point, we will progress to allowing AI to partially automate some tasks—perhaps those that are less risky, such as collecting diagnostic information and adding it to the alert, so that the incident responder automatically receives both together. We may even automate some parts of our system; perhaps within a staging environment that isn’t production-critical and, as such, is a good place to validate that the automation is working as expected. Over time, as our AI and ML capabilities develop and our trust in them grows, we may eventually reach full AI-driven automation, with no human involvement at all—the AI will simply detect the fault and resolve it. 

    Pillar Two: Predict and Prevent 

    Typically, when assigned an incident, a responder will take note of the incident’s priority. But they will also rely on their common sense and past experience in deciding how to respond. So, for example, if a P3 incident had recently escalated to a P1 in 30 minutes and a similar P3 incident now appears again, our responder will probably prioritize it, even over other incidents, to avoid a similar escalation.

    While this is one of the advantages of human logic, the flip side is that the IT environments in which we work are vast and complex—no human can possibly remember every potentially relevant detail. What’s more, if a different responder dealt with the earlier incident in our example, the current responder would not have the same insight. What is needed is an element of enrichment that can input leading indicators into the platform for the next operator to see and act upon to prevent an incident.

    This is where AI can help. 

    By identifying patterns in the data haystack—linking different incidents to the same root cause, for example—AI not only generates insights that help resolve these incidents faster but can also predict a certain outcome. This is done by anticipating a future chain of events so that appropriate preventative action can be taken before the event occurs. For now, this preventative action may be manually taken by the operator, but at some point in the future automation may enable the problem to be identified and fixed by the AI with no human intervention before it results in an outage.

    Pillar Three: Democratize

    The last pillar that will help us achieve full autonomy in IT Ops is democratization of AI. AIOps should not just be accessible to advanced, mature organizations with an army of data scientists working behind the scenes. Just as anyone can accomplish almost anything online these days with just a few clicks, any organization should be able to adopt AIOps and enjoy its benefits! 

    Democratization is about making relevant information accessible to anyone who needs it, when they need it, and in a manner that is easily applicable and actionable. AI should be accessible through simple user interfaces, with no need for coding; the benefits of machine learning should be available to people that don’t (need to) understand machine learning models. 

    This democratization can be achieved by making AIOps platforms simple and accessible to all—from the admin to the user. For example, an AIOps platform might automatically suggest how admins should prioritize incidents for operators, to make triage easier and provide instructions that even a non-expert can follow. The more such features are built into AIOps platforms, the more democratized AIOps will become.

    Data volumes will continue to grow exponentially in the coming years, and the only way to eventually conquer the tyranny of overwhelming data will be to reach autonomy in IT Ops. 

    If you’re an AIOps vendor, keep the three pillars in mind so your customers can continue to progress towards autonomy.  If you’re an IT Ops leader or tooling architect, and IT Ops autonomy is part of your vision, make sure that your AIOps vendors’ platforms and tools have been architected with these pillars and their principles in mind. 

    How far are we from achieving autonomy? Only time will tell. But one thing is for sure—we will get there, and if the COVID-19 pandemic has taught us anything it’s that we may be seeing machine-augmented teams sooner rather than later. 

  • 3 Building Blocks for Enterprise Digital Transformation

    3 Building Blocks for Enterprise Digital Transformation

    DevOps has gone from a new idea to a buzzword in every boardroom over the last decade. With society increasing its reliance on digitization and pushing their preferences for speed and convenience, enterprises of all types have had to change their business culture to help their teams to work together with greater speed and autonomy.

    While the pandemic has undoubtedly heightened the importance of DevOps in the workplace, its widespread use predates the last few years. HGS Digital research from as far back as 2018 found that only 3% of software development businesses had never heard of DevOps. The 2021: Future of the Atlassian Ecosystem report revealed that 54% of organizations have implemented a DevOps strategy—up from 48% in 2020. A further 27% of respondents are hoping to adopt a DevOps strategy in the next three years.

    Indeed, DevOps is now mainstream, and the uptick in job listings for ‘DevOps engineers’—as well as enterprises’ quickness to tout their ‘experienced DevOps teams’—is perhaps further evidence of this. However, some of these postings and sentiments can also be considered red flags as they suggest that organizations are applying DevOps practices to isolated teams. Or worse, they are simply labeling traditional engineering roles with the DevOps label rather than implementing a complete mindset shift that applies to the entire organization and enables it to release reliable products at a much faster pace.

    With this in mind, here are three key building blocks for enterprises looking to avoid common downfalls and effectively scale their DevOps strategies. These will help them change the way they work and think across all aspects of development and operations.

    Changing Culture and Empowering Employees

    Everyone needs to be on board for DevOps to be successful, and buy-in is strongly dependent on the business culture they are working within. For example, Gartner predicts that through 2022, 75% of DevOps initiatives will fail to meet expectations. And the reasons why have to do with issues around organizational learning and change.

    To put it another way, if employees don’t understand DevOps’ deeper thinking and benefits both to them personally and to the wider group, they will naturally be resistant to the change. It is, after all, an entirely new way of thinking for many departments, especially operations teams who traditionally reduce risk by minimizing change and taking things slowly. DevOps encourages them to speed up with minor changes delivered through continuous integration operating in a fail-fast and often fail-forward pipeline.

    Therefore, educating employees is vital for success. In particular, organizations should give staff a clear idea of how it will feel to work differently. Shared definitions of success, shared vocabulary and shared visibility of the work to be done is key. With that—and along with sharing actionable insights that teams can apply to their work and then observe the results—no matter where they are in the pipeline, the real value of DevOps can be achieved.

    The reality is that people are the most critical component of any DevOps transformation, making their commitment to the cause non-negotiable for success. Despite the tech-heavy nature of software development, soft skills like communication, collaboration, flexibility and decision-making must be prioritized as much as security, coding and infrastructure knowledge. It’s imperative that all businesses look to adopt a DevOps mindset and implement these traits into their culture in the future.

    Applying Four Fundamental Processes to the Foundation of DevOps

    That said, a successful DevOps transformation can’t happen unless enterprises step away from traditional IT’s ‘trust but verify’ methodology. Instead, they must incorporate a set of essential processes that empower everyone to work more efficiently and effectively toward achieving faster, high-quality software delivery to suit the end-user’s needs. These fundamental processes include collaboration, automation, continuous integration and continuous delivery (CI/CD) and monitoring.

    While collaboration has been one of the most significant rallying cries for enabling globally distributed IT teams to align their goals and meet their objectives, automation has also been highly crucial. Many of those ingrained in DevOps environments will tell you that automation is what makes DevOps highly valuable. It helps teams achieve high levels of efficiency and allows employees to focus more on complex problems and less on repetitive tasks. When a company’s automation strategy is aligned with its broader business goals, it can help deliver value at scale.

    But as is the nature of software development, problems can arise. And this is where continuous integration and delivery come into play; it forces developers to merge their source code updates with others far more frequently than they might otherwise choose to. Operations teams can ensure they’re not creating a bottleneck with manual testing to get releases out to users. The approach means smaller changesets; helping to identify, target and resolve bugs more efficiently. This increases the flow of work, the quality of releases and helps to avoid nasty problems at the end of the development cycle.

    But there’s no point in having a speedy, heavily automated CI/CD process with a genuinely collaborative culture if enterprises don’t take the time to measure what’s working and assess which actions need to be improved. This is why an effective DevOps plan needs to be built on the mantra ‘implement, measure and improve’. After all, DevOps is about maximizing value and efficiency through increased visibility of work and constant feedback. Measuring real-time metrics becomes vital to ensure business’ transformations are in line with their objectives at all times. If enterprises stick to these fundamentals, they’re well on their way to a successful DevOps strategy with complete buy-in.

    Adopting the Right Tools for Transformation

    As teams grow and DevOps processes scale, predictable performance is imperative. Therefore, the enterprise’s tools must be available around the clock and should perform at a high level to support their collaborative pipeline.

    But using DevOps tools doesn’t automatically mean organizations are applying DevOps principles. The value of tooling is limited by businesses’ commitment to using them holistically with the sort of culture and processes mentioned above. It is this combination that allows them to transition how they develop and deliver their products. With all that said, it’s important to understand how important it is to adopt the proper toolchain—and the wide variety of choices and combinations available—this is why it’s easy to understand why decision-makers might become overwhelmed.

    After all, there’s a lot to keep in mind. They need to choose tools that step up their collaboration, keep context-switching to a minimum, aid automation, encourage visibility and allow for effective monitoring. But there’s also the risk that they’ll pick the solutions that they feel most comfortable with, which can be counterintuitive to transformation. This approach can lead to a tool binge, bringing on too much technology where integration isn’t a priority, and leaving enterprises with nothing that does the job they need.

    With this in mind, enterprises need to take the time to consult on which tools are right for them while constantly keeping the fundamentals of DevOps in mind. To illustrate just how valuable this is, DORA and Google Cloud’s 2019 State of DevOps report found that the highest performing engineers are 1.5 times more likely to leverage easy-to-use tools. Meanwhile, failure to do this is perhaps a primary reason why a separate study from the DevOps Institute uncovered that the DevOps transformation journey is still arduous for more than 50% of respondents.

    Taking the Next Step Toward DevOps

    A successful DevOps transformation rests on in-depth understanding, mass buy-in and an organization-wide adoption of fundamental principles and practices. When this agile mindset is applied to the people, processes and tools that make up an organization, enterprises can drive significant change across their teams while increasing their products’ reliability, stability and security.

    It’s always important to keep in mind who DevOps is designed to benefit the most—the customer. This mindset is particularly valuable in today’s landscape, where businesses and consumers alike are flush with more choice than ever. With speed and convenience now at the forefront of most people’s purchasing decisions, DevOps can play a huge role in helping organizations strive for greatness and offer the customer something they didn’t even know they needed. Using the above building blocks, organizations of all sizes can dramatically improve their customer experiences and work more efficiently to quickly get the right products into their hands.

  • Google Donates Sqlcommenter to OpenTelemetry Project

    Google Donates Sqlcommenter to OpenTelemetry Project

    Google announced today that Sqlcommenter, an open source project that provides access to an object-relational mapping (ORM) auto-instrumentation library, is being merged with OpenTelemetry, an open source project for building agent software to instrument applications.

    Nimesh Bhagat, a product manager for Google, said Sqlcommenter is designed to make it simpler to correlate application and database telemetry. OpenTelemetry provides a mechanism for collecting logs, metrics and traces that is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The challenge organizations using OpenTelemetry to collect that data will face is the need for tools to query that data in whatever platform it ultimately resides, noted Bhagat. OpenTelemetry previously lacked a common standard by which application tags and traces can be sent to databases and correlated with an application stack. Google envisions that IT teams will either employ Sqlcommenter to explore data or access via platforms such as Cloud SQL Insights or a third-party application performance management (APM) platform.

    Google made Sqlcommenter available as open source software earlier this year. It is designed to enable object–relational mappings (ORMs) to augment SQL statements before execution by, among other things, providing access to comments containing information about the application code that caused its execution.

    Sqlcommenter also allows OpenTelemetry trace context information to be propagated to the database, enabling correlation between application traces and database query plans that ultimately should make it easier to, for example, determine the root cause of a performance issue involving a microservices-based application making calls to a database. Sqlcommenter is primarily designed to be employed by full-stack developers that have some familiarity with how to use SQL to query data residing in an observability platform, noted Bhagat.

    Specifically, Sqlcommenter is designed to provide deeper insights into the performance of backend databases by simplifying the process of correlating slow queries with source code. It currently supports Python, Java, Node.js and Ruby languages along with ORMs such as Django, Sqlalchemy, Hibernate, Knex, Sequelize and Rails.

    As the number of applications that are instrumented using open source software continues to steadily increase, the amount of data that a DevOps team could potentially query also will steadily increase. Cloud storage platforms have made it easier to store that data at an affordable cost but the next challenge is clearly going to be finding a way to surface meaningful insights from all that data. Undoubtedly, achieving that goal will require everything from query tools to machine learning algorithms capable of identifying patterns at a level of scale that would not otherwise be possible.

    In the meantime, DevOps teams can look forward to being able to attain a level of observability that previously could only be achieved at a cost that tended to limit the number of applications organizations would instrument. At a time when the dependencies between applications increases with each passing day, that level of observability will soon prove indispensable.

  • Do’s and Don’ts of Phased Rollouts Software Delivery

    Do’s and Don’ts of Phased Rollouts Software Delivery

    When it comes to app stability and user experience, the “move fast and break things” approach is a recipe for disaster. To protect brand reputation and to stay ahead of evolving needs and customer expectations, leading Silicon Valley companies’ software engineering teams have adopted phased rollouts, a software development method that involves the gradual implementation of a new feature to a smaller cohort of users at one time rather than all users at once.

    Phased rollouts help companies maintain the highest-quality customer experience (CX) for their users while making changes and software updates. When significant features and updates aren’t rolled out to production all at once, the organization’s software engineering team doesn’t have to deal with all the potential implementation issues at once. With phased rollouts, engineering teams catch issues early in the process so they can quickly fix them and resume the rollout of the experiment, which prevents issues from impacting millions of users down the road.

    While companies everywhere try to emulate the app development techniques of these Silicon Valley tech leaders, many are unable to achieve successful phased rollouts at scale. What they often don’t realize is that before even implementing a phased rollout approach—whether it be to 5%, 10% or 15% of their user base—there are a few steps that must be taken to ensure success. Below, I’ll walk you through the do’s and don’ts of a successful phased rollout plan.

    Do: Release Stability Management

    Companies must drive consistently stable app and web experiences in order to succeed in the market today. Therefore, it’s critical to continually monitor the stability of releases throughout development and testing to catch problems early and decide when to roll features out to users. By measuring and monitoring app stability in real-time, release managers inject accountability, predictability and complete peace of mind into the organization. Taking control of release stability management helps engineers deliver stable applications to drive world-class digital experiences.

    On top of that, software development teams must constantly improve mobile and web apps with new features and capabilities that better enable users to accomplish what they want to do. These kinds of improvements give companies the competitive edge they need to survive in today’s competitive business environment. In addition, monitoring release stability provides engineering teams with real-time visibility into every issue impacting application stability and CX. This level of visibility is critical for quickly diagnosing and understanding the root cause of errors so that engineers can roll out fixes before users are negatively impacted.

    Do: Testing and Early Access

    For a phased rollout, engineering teams must run A/B tests and quality assurance (QA) testing. A/B testing (also referred to as split testing or bucket testing) is a method of comparing two versions of a web page or app against each other to determine which one performs better. This includes trying out features like different screenshots, videos and even different app icons to measure which format users like the most.

    QA testing ensures that the organization delivers the best products or services possible by detecting usability issues that might not be technical. It tests for any problems that impact the end-user experience, from poor navigation to slow load times or unclear web copy.

    With phased rollouts, you must also provide early access to new features for select users to collect feedback and stamp out critical issues before launch. Many software products collect user feedback by including a small checkbox somewhere asking users to help make the software better by sending insights into how they use the application. In addition, social media has become an undeniable avenue for collecting candid user feedback. We’ve all seen it—when a platform changes its interface for certain users before others, users will post on social media sites about whether they are pleased with the new update or not. Historically, there has been backlash when significant changes are made and the company then has to decide whether to keep, change or cancel the new changes to the platform.

    As mentioned above, with an app stability monitoring tool, you can see if new features or experiments are introducing new stability issues and make decisions about pulling back or rolling out to a larger subset of users. By conducting testing and allowing early access to only a specific cohort of users, software engineering teams can balance release speed with application stability to create the best experience possible.

    Do: Allow for A Staging Period Before Going into Production

    As you launch new features, you’ll want to test them in a production-like environment after they go through QA testing. This is called staging, and it has become a popular concept with Silicon Valley companies. Staging is a near-exact replica of a production environment for software testing. Staging is meant to test code, builds and updates to ensure quality under a production-like environment before application deployment.

    A production-like environment is similar to when the app is released into production for users, but it’s not public yet. It provides an accurate view of how the update or new feature will operate and look in a production environment to avoid glitches that are only noticeable in production. Staging can include manual tests or, in some cases, engineering teams use automated software testing, which is more costly to implement and may only be attainable for larger enterprises.

    In this phase, software engineers should conduct additional tests and experiment with the new features to ensure that these capabilities—when released to production or to the larger user base—will not compromise the app’s stability. The last thing you want is for a new feature to break the user experience.

    Don’t: Try to Fix Every Bug or Error as it Occurs

    Since engineering teams often don’t have the bandwidth to fix every coding error that should arise, they need to prioritize and fix the errors that impact customers directly in order to protect revenue. Although organizations should have visibility into every error or bug, not all of them are worth fixing. This causes the signal-to-noise ratio that plagues developer efficiency.

    This is where app stability management plays an essential role. However, many of the app stability monitoring solutions on the market today can help with identifying errors, but leave it up to development teams to figure out which ones need to be prioritized. Engineering teams have to quickly determine which errors are the most urgent to fix, so they need to have reliable tools such as an app stability and error monitoring platform to provide them with visibility to make these important decisions in real-time.

    Phased Rollouts for Progressive Delivery

    Phased rollouts allow software engineers to continuously optimize the digital experience, without impacting the whole user base and negatively affecting the brand. Once mastered, phased rollouts can be used as part of a cutting-edge progressive delivery approach to software engineering and development.

    A progressive delivery approach includes shipping code (sometimes unfinished code) to a production environment multiple times a day, exceeding many companies’ risk tolerance. In order to do this successfully, engineers must have control over who can ship releases and when, and which users can access certain features and updates at what time. Phased rollouts provide a way to shield the majority of customers from the failures that occur at those rapid release speeds. To ship new software releases faster, engineers have to spend less time on QA and testing, but that doesn’t mean they should be overlooked completely. Then, by delivering the new release to a small subset of users first, you can manage risks more effectively and iterate accordingly.

    It’s important to call out that if the engineering team is making minute changes that no users will see or care about, you don’t need to use a phased rollout approach. That said, if it’s a significant cosmetic or operational change, such as updates to the UI, then phased rollouts are crucial. A phased rollout strategy is key for making software or app updates at scale that are necessary for a successful launch in today’s highly competitive market.

  • Driving E-Commerce Transformation with Microservices

    Driving E-Commerce Transformation with Microservices

    It’s no surprise that COVID-19 has accelerated digital transformation, but just how fast these changes have occurred is incredible. According to recent research, companies have accelerated their share of digital or digitally enabled products by up to seven years.

    These changes are impacting every sector in every region around the world, but they are being felt acutely by companies that sell their solutions online—which I imagine includes your company. To thrive in the new normal of digital commerce, businesses need to be supported by a fast, flexible technology architecture that can adapt to new products and go-to-market strategies and can be spun up on the fly. The best—and really, the only—way to do that is with a microservices architecture.

    Why Monolithic Architectures are Uniquely Bad for Commerce

    While monolithic architectures can be simple to develop, test, and deploy, their limitations, especially in the age of digital, quickly become apparent. Software becomes too large and complex to make changes quickly, or correctly. Bugs can be a disaster, and monolithic architectures are often a barrier to adopting new technologies since any change requires significant time and monetary investment to update the entire application.

    These limitations are even more glaring in the age of subscriptions. In today’s market, if you are selling products or services online, you are likely offering subscriptions. In fact, 95% of companies offer at least one subscription-based product. For subscription commerce businesses, the need for microservices architecture is even more critical. A recurring revenue strategy requires frequent adjustments and easy integration with other systems, requirements that a monolithic architecture often can’t accommodate.

    Scale Is the Critical Factor

    One of the greatest benefits of a microservices architecture is the ability to scale quickly. Monolithic architectures are much more rigid because everything has to be scaled at the same time. In contrast, a microservices architecture allows organizations to scale individual components of an application to meet niche use cases. Moreover, because microservices give you the freedom to update individual components, organizations can make changes without any interruption to broader systems. In other words, with a microservices architecture, developers are able to upgrade the airplane while it’s in flight.

    There’s internal scale, but external scale through ecosystems is also an important consideration. Recent research found that 56% of executives believe engaging in ecosystems is the most effective way to access new markets, while 55% believe that the new capabilities gained from partnering are essential to building new capabilities. Ecosystem participation—whether as an operator or a participant—requires timely integration and development, a reality that monolithic architectures just cannot support.

    Microservices as a Recruitment Tool

    Innovation and digital transformation aren’t just business needs. They can also be one of the most powerful recruiting tools available to draw in the best talent. The job market is rife with opportunity for prospective employees—especially now, during the Great Resignation, where a significant number of employees are looking to change jobs—and businesses need to embrace every advantage they can to attract top talent.

    Legacy technology can have its place, but it can also be a red flag for job candidates. Most developers want to join an organization that values innovation and gives them the freedom to build unique products and solve meaningful business problems, which microservices development offers. A microservices architecture can also make onboarding new talent easier, since developers aren’t obligated to learn every facet of the architecture. It’s a win for new employees and existing organizations alike.

    Driving Transformation Through Microservices

    The needs of any given business are always changing, but digital and subscription commerce businesses have proven to be extraordinarily dynamic. Monolithic architectures of the past simply aren’t equipped to power digital transformation or support commerce businesses when they emerge on the other side. Microservices may come in a smaller “package,” but they will deliver much bigger benefits to your business in the long run.

  • Accelerating Software Testing

    Accelerating Software Testing

    Yves Junqueira is the CEO of YourBase, a software testing accelerator for large codebases. He talks with Alan Shimel about the company, accelerating software testing and the impact on developer productivity. The video is below, followed by a transcript of the conversation.

    Alan Shimel: Hey, everyone. Thanks for joining us here on another segment for our TechStrong TV. My guest this segment is a new company, first-time guest on our TechStrong TV, Yves – and I’m gonna really mispronounce his last name, Junqueira. I’m supposed to have that rolling R, and I just don’t do rolling R’s. Yves, I’m sorry, man. Welcome to TechStrong TV.

    Yves Junqueira: Thank you, Alan. It’s great to be here. You did it good.

    Shimel: Not so good. You know what? In case your mom or your family’s watching, say your name correctly for ’em.

    Junqueira: Yves Junqueira.

    Shimel: Junqueira. Okay. We’re gonna call you Yves, and we go from there.

    Junqueira: Sounds perfect.

    Shimel: So, looking from your background, are you up in Seattle, Toronto? Toronto?

    Junqueira: That’s Seattle.

    Shimel: Seattle. Yes, the mountains. And you know what? Welcome. I’m gonna let you give a little introduction to your company, and you’re a cofounder of it, but go ahead.

    Junqueira: Gladly, yeah. Our company’s called YourBase. We build developer tools for very large code bases. Specifically, we are a software-testing accelerator for large code bases. I used to work at Google for almost ten years, and my cofounder, John, was a principal engineer at Amazon, at AWS. And, yeah, we’re excited to – I’m very excited to be here, Alan.

    Shimel: Absolutely. Well, I’m excited to hear about what you’re doing. So you’re a code accelerator?

    Junqueira: We are a software-test accelerator, yeah.

    Shimel: Software-testing accelerator, excuse me. What exactly is that?

    Junqueira: Yeah, so it’s very important when you’re building software that you actually know that the software is working. You usually have very large teams that’re creating code, and you don’t want one developer to be stepping on each other – on the other’s toes. And so you need a lot of testing to make sure that when you make changes to the software, it’s still working. But as you’re adding new features, you’re adding new tests, and you add new features, and you add new tests, and over time the number of tests in your code base can grow very, very quickly, and it can become a very big problem for developer productivity and developer happiness, because it can take a very long time to actually test code. If you’re making even a small change sometimes in a large code base, it can take hours to ensure that that change works, to run all the tests. So that’s what we help with.

    Shimel: So you’re automating testing. Is that fair?

    Junqueira: We automate the test selection of testing. So, yeah, we help run the tests, but most importantly we help select what are the right tests that are safe but that have to be run for each change. Without having to test the entire world, we use a process called – we use tracing of code to build a dependency graph to then select the right test to run and then just make things more productive for developers.

    Shimel: Sure. Of course, our audience is a very technical audience, familiar with QA and testing and developing. So you’re running a trace route on the code as it’s entered into Git or – because it’s pre-deployment code, right?

    Junqueira: That’s right. It’s pre-deployment code, yeah.

    Shimel: So as the code is put into Git or some other sorta repository, you select what’re the right tests to run and run it on the code. So this all takes place within the repository like a Git-ops kind of thing?

    Junqueira: Yeah. It can take place in the repository connected with a CI, with a continuous-integration tool, or it can also happen during local development. You can have multiple developers working at the same time on the same code base but perhaps making slight – different code changes. And with your base, they are testing their code but also using their colleagues’ testing to – so that they don’t have to repeat the test that their colleagues already ran for the same code base. So they only have to test exact the small changes that they made on their code.

    Shimel: I love it. Sounds great. And then I don’t wanna use the buzzwords, but is it some sort of AI/ML that is picking what’s the right code – what’s the right test to run for this particular code for these particular changes?

    Junqueira: That’s an important question. It’s not AI. We use just simple data structure and algorithms that we collect when doing tracing. And this is actually important, because if we were using just AI or any sort of statistical approach, we would only get a limited confidence that we are testing the right things. So imagine that you’re a financial institution, right, or a hedge fund, where tests are very important. If you don’t catch certain bug, it can cause millions of dollars in losses, right, in lost revenue, for example. So it’s very important that you actually detect the correct regressions in your code, and if we were using just a simple AI approach, we would only, let’s say, have 90 percent confidence that we were running the right tests in your code. That is not enough. So our customers need assurance. They need full confidence, so we use an approach that is a little bit more intrusive, let’s say, because it’s detecting every single function called that is happening during the tests. So it has a little bit of a performance impact during the initial test. But on the other hand, it just provides full confidence rather than being 90 percent confidence that an AI approach would have.

    Shimel: Got it. I mean, big companies using this, small companies, anyone? Who’s the target user here?

    Junqueira:Fortunately, we can scale from small to very large code bases. So we have customers that are very small, just small startups that are getting started but that really want to have the fastest possible development process. But we also have very large code bases of very large companies, public companies, Fortune 500s, that want to keep moving fast. It’s difficult. Imagine if you have a software product that really needs to evolve quickly. Customers these days demand that software developers – that software companies evolve their products quickly. The customer needs are changing fast, and so they need to be able to make software changes very quickly and do it at – have high velocity with the most productivity possible. So what we allow those companies to have is even though they have a very large code base, the larger companies, they can be almost as productive or as productive as if they had a small code base, and this is really magical. This is something that we’re very proud of providing to these companies.

    Shimel: Absolutely. You know what? People always wanna know this, and I realize it might be coming outta left field. Interesting. You worked at Google ten years. Your cofounder’s an AWS guy. How did that come together?

    Junqueira: Yeah. It’s actually a funny story. We met at a random coffee shop. After I had left Google, I was working at random coffee shops here in the outskirts of Seattle, and I had my laptop with me with a bunch of developer stickers there like Linux and Python and GOAL like developers often have. And John had the same, and so we just looked at each other’s laptop. We were at this coffee shop here, and we – okay, we should chat, ’cause we us a lot of the same technologies. So we started talking, and over time I invited John to come to my weekly table games at my house. I host a game night every week, and John started coming, and I started talking about my project, and he was fascinated by it, and he said that software testing and developer productivity are a huge thing for him. And over time, he decided to join. I’m very lucky that he joined me at YourBase.

    Shimel: Love it. And then talk a little bit here about going to market. How’s it been so far?

    Junqueira: Yeah. We spent a long time perfecting the technology. This is a very tricky software. If you get things wrong, you may end up – your customers may end up shipping bugs. My customer’s gonna end up shipping bugs that should have been caught by the tests. But if I don’t decide to run a test or if I don’t give them right insights, they might actually introduce bugs and be less reliable than they were before. But we started just last month. We launched our first public beta now, and we’re very happy about that. So we have a number of design partners that are working with us on this initial release, and people – customers are very happy. The feedback has been fantastic. So last month we launched it to the outside world, and we made a public beta. Now people can just download it from just going to the website, yourbase.io. Anyone can download and test it. We currently support Python as – for code bases. Our technology has to be implemented on a language-by-language basis. So for now we support Python, but we’re going to support Ruby sometime soon. And we’re very excited. The reception from people has been amazing. It’s really fun to be at this stage of the company, where we actually are ready to talk about it to the outside world.

    Shimel: Excellent, man. It is. Having founded more than one company, more than four companies, five companies, it is an exciting time, and I wish you nothing but great luck and success with this. I want to get into the kinds of testing you’re doing, functional testing, unit testing. Is it selenium kinda stuff? What are the kinds of tests you’re doing there?

    Junqueira: That’s a great question. We are very good at tests that don’t cross process boundaries for now. This can be sometimes unit tests and functional tests. We don’t currently support UI tests yet, in part because there are good companies out there that’re already serving this market, and there was just nobody that was focused on the backend and in unit testing and functional testing. But this is a big space, and we’re just getting started. We have feedback from folks that we – they want support for more things, so that’s something that we’re excited to work on.

    Shimel: Excellent, man. Yves, I wanna thank you for coming on and telling folks about YourBase here on TechStrong TV. Come back. Keep us posted on your progress, and best of luck.

    Junqueira: Thank you so much, Alan. It was great to be here, and, yes, absolutely, I would love to come back.

    Shimel: My pleasure. Say your name for us one more time so I don’t – just after I mangle it, at least they got it right.

    Junqueira: Yves Junqueira.

    Shimel: Junqueira. Yves, thank you, man.

    Junqueira: Thank you, Alan.

    Shimel: Oh, the website, yourbase.com?

    Junqueira: Yourbase.io.

    Shimel: IO, yourbase.io, Y-O-U-R-B-A-S-E-dot-I-O. This is Alan. We’re gonna take a break. We’ll be right back with our next guest.

    [End of Audio]