devopsy.kinsta.cloud

Category: DevOps Practice

  • What It Really Takes to Build a Reliable App

    What It Really Takes to Build a Reliable App

    What makes an app reliable? If you ask most IT professionals that question, their minds immediately go to uptime. That’s not surprising; after all, it’s fashionable for software and infrastructure vendors these days to boast about how many nines of uptime they guarantee.

    To be sure, uptime is one component of delivering a reliable application experience for developers and end-users. But it’s only one. In order to build reliability into your application patterns, you must think beyond uptime alone.

    Keep reading for an overview of all the factors that go into achieving reliability.

    What Is Reliability?

    A reliable application is one that meets the needs and expectations of everyone it serves. While those needs and expectations will vary from case to case, reliability is characterized by the ability of an app to:

    • Be accessible when needed.
    • Respond within the time frame needed.
    • Be updated or modified as needed.
    • Provide security and privacy to the extent needed.
    • Meet the needs and expectations not just of end-users, but of everyone else who helps create or support the app.

    Maximizing Reliability for Cloud-Native Apps

    Achieving these features for modern, cloud-native applications requires a multi-pronged approach. The following are the core considerations to address in order to build reliability into an app pattern:

    Uptime

    We’ll start with uptime, which is the most obvious aspect of reliability. Uptime refers to the amount of time that your application is available, at least to some extent (as we’ll discuss below, merely being available does not necessarily mean that the app satisfies other requirements of reliability).

    Maximizing uptime typically involves solutions such as distributing workloads across different servers or data centers (so if part of your infrastructure fails, others remain available) and using automated failover to move a workload to new infrastructure if it fails in one location.

    Culture and processes are part of the picture, too, since you need to make sure your team is prepared to respond quickly to any incidents that arise, in order to minimize downtime.

    It’s worth noting, by the way, there can be such a thing as too much uptime. If you reach the point where adding more uptime doesn’t deliver any additional value to your organization, then it’s probably time to stop. Adding another nine to your availability, just because you can, might be a poor use of resources. So, be sure to factor this part of the equation into your uptime calculations and strategy.

    Performance and Efficiency

    In order to meet stakeholder needs and expectations, an application must not only be available, but also able to respond within the time frame required by those stakeholders. A website that is up 100% of the time but takes 20 seconds to load each page is not a very reliable website.

    This is why performance testing and monitoring must be built into your application delivery chain. You should also ensure that you rightsize infrastructure (for example, by choosing the right types of cloud instances) in order to guarantee that your applications have enough resources available to perform adequately. (Of course, this priority must be balanced with cost-control.)

    Accessibility and Usability

    An application that is available all of the time and responds quickly may still fail to meet stakeholder needs if it is difficult to work with due to poor design.

    From an end-user’s perspective, user-experience testing is the most obvious way to help mitigate the risk of poor accessibility or usability within your applications. But keep in mind that you also need to think about usability from the perspective of your own team. Make sure that access-control systems, infrastructure architectures and documentation can be navigated easily. You don’t want an application or a delivery chain that is too complex for your team to be able to support effectively.

    Security and Data Privacy

    A reliable application is a secure application. There are many obvious ways to help secure applications, such as using vulnerability monitoring tools. But remember security is not just about finding vulnerabilities as they arise or deploying certain tools. You must also build security into the design of your infrastructure, processes and culture.

    Backup and Recovery

    You don’t need to be a seasoned IT professional to know that backing up data is important for achieving reliability. When something goes wrong–which it occasionally will, despite your best plans and efforts–having a backup in place often means the difference between experiencing a blip in service and a major catastrophe.

    The thing that can be easier to overlook is the importance of having a disaster recovery plan in place as well as a backup routine. Simply backing up your data isn’t enough; you must also know how you’re going to restore it to new application instances quickly if disaster strikes. If, for example, all of your cloud-based virtual machines go down, what is the exact process you’ll follow for standing them back up? Identifying the steps ahead of time–and, ideally, scripting them so the recovery process can be as automated as possible–is crucial.

    Also important is quantifying how often you need to back up data, which is determined by a metric called Recovery Point Objective (RPO), and how quickly you need to be able to recover it to avoid serious business disruption–determined by Recovery Time Objective (RTO). Along similar lines, error budgets, which define how much time you can tolerate your production systems being down, can help you define the available time monthly for making system improvements.

    Modifiability

    Few apps are deployed and then never modified again. Instead, they must be updated or scaled constantly to meet changing needs and demands. That’s why designing an application pattern that can be easily modified is critical for achieving reliability over the long term.

    In practice, modifiability entails having the right tools (such as automated deployment solutions) and the right processes (such as clear feedback loops between ITOps and developers) in order to decide efficiently what modifications to implement. It might also involve taking advantage of infrastructure technologies (such as containers) that make it easy to update a running application without imposing downtime on users.

    Conclusion

    Reliability is more complex than it may seem. No matter how many nines are included in your SLAs, you are not guaranteeing true reliability unless you also address requirements such as performance, usability and modifiability.

    This sponsored article was written on behalf of Eplexity.

    — Chris Tozzi

  • Load Balancing: Round Robin May Not Be the Right Choice

    Load Balancing: Round Robin May Not Be the Right Choice

    Based on our experience, we believe Round Robin may not be an effective load balancing algorithm, because it doesn’t equally distribute traffic among all nodes. You might wonder, how is this possible?

    How Does Round Robin Algorithm Work?

    Round Robin algorithm sends requests among nodes in the order that requests are received. Here is a simple example.

    Let’s say you have three nodes: node-A, node-B and node-C.

    • First request is sent to node-A.
    • Second request is sent to node-B.
    • Third request is sent to node-C.

    The load balancer continues sending requests to servers based on this order. It makes it sound like traffic would get equally distributed among the nodes. But that isn’t true.

    What is the Problem with Round Robin Algorithm?

    Figure 1: Load Balancer with 2 nodes (Round Robin).

    Let’s pick up a simple example: Let’s say you launched your web application with a load balancer and it has two nodes (node-A, node-B) behind it. The load balancer is configured to run with Round Robin algorithm, and sticky-session load balancing is enabled. Let’s say 200 users are using your application currently. Since the Round Robin algorithm is enabled in load balancer, each node will get 100 users requests.

    Figure 2: New node added. Newly added node gets less traffic in Round Robin algorithm.

    A few minutes later, you are adding node-C. Now, an additional 100 users start using the application. Since it’s Round Robin algorithm, load balancer will distribute new users’ requests equally to all 3 nodes (i.e., 33 users request to each node). But remember node-A and node-B is already processing 100 users requests each. So now node-A and node-B will end up processing 133 users requests each (i.e., 100 original users requests, plus 33 new users requests), whereas node-C will process only 33 (new) users requests. Now, do you see why Round Robin isn’t equally distributing the traffic?

    In the Round Robin algorithm, the older nodes in the pool will always end-up processing more requests while newly added nodes will end up processing less amount of traffic. The load is never evenly distributed. For maintenance, patching and installation purposes, you have to continually keep adding and removing nodes from the load balancer pool. If you instrumented auto-scaling in place it gets worse, because in auto-scaling nodes are more dynamic. They get added and removed even more frequently.

    What Algorithm to Use?

    Figure 3: New node added. Traffic is evenly distributed in Least Connections.

    There are a variety of load balancing algorithms: Weighted Round Robin, Random, Source IP, URL, Least Connections, Least Traffic, Least Latency. Given the shortcoming in Round Robin, you can consider trying other choices. One choice you may consider is: Least Connections algorithm. As per this algorithm, the node which has the least number of connections will get the next request. So, as per our earlier example, when 100 new users start to use the application, all new user requests will be sent to node-C. Thus, load will be equally distributed among all nodes.

    — Surya Baby

  • Serverless and GraphQL: Build and Launch at the Speed of Thought

    Serverless and GraphQL: Build and Launch at the Speed of Thought

    Websites and apps are now, often, one and the same. Traditional development methods for an online or downloadable app can take months, given the backend development aspects that need to be managed—infrastructure provisioning and management, security, scalability, consistency and more, particularly for companies that have users around the globe and potentially very high usage demands.

    In addition, once the app is built, it can take several days to launch and ensure global availability. Once launched, the app can require ongoing maintenance and supervision to ensure availability and performance, as well as constant fine-tuning of the balance between over- and under-provisioning to manage costs.

    Serverless computing, when combined with GraphQL APIs, is the Holy Grail of application development. These technologies make it possible for developers to build and launch apps lightning fast by removing server provisioning and management concerns. Developers need only worry about building a compelling app with usability features that optimize engagement. Additionally, with serverless and GraphQL, an app can essentially be launched to a global user base within hours or even minutes, rather than days, while avoiding the need to manage servers. Let’s take a look.

    What is GraphQL?

    GraphQL is an API standard for building modern, data-driven applications based on the widely known HTTP Protocol. First created by Facebook in 2012, GraphQL was key to the social media giant shifting its mobile app from HTML5 to a native mobile app. Based on its success, Facebook made GraphQL available to the public as an open-source project in 2015.

    With GraphQL, you define an agnostic Domain Specific Language (DSL) around your data structure in terms of:

    • The programming language you use. As an API standard, GraphQL is open for implementations in many languages, such as JavaScript, Java and C#. You can essentially use the standard with your language of choice.
    • The database engine you use. GraphQL is only concerned with having the data ready, rather than the source of the data and where it is coming from, so you have the freedom to implement a GraphQL API with almost all database engines.

    What’s All the Fuss About Serverless?

    Serverless represents a major paradigm shift in the way we create and run web services and apps. Serverless computing operates in the cloud, freeing developers from the burden of thinking about back-end tasks, such as managing infrastructure, scaling servers and provisioning capacity and resources.

    Below, I’ve listed some specific benefits that serverless provides, looking specifically at serverless databases—many of these advantages are part of other serverless offerings as well. The overall impact is that with serverless technologies, developers only pay for what they use, or what the app uses depending on user demand. Developers can scale up to virtually infinite disk space and compute loads in seconds when traffic spikes hit, and scale back down when users go offline. The NoOps aspect of serverless means maintenance costs and burdens are removed. Additionally, global availability zones allow for replication between different locations and allow you to keep data where it’s needed—close to the user to minimize latency. Built-in encryption means developers can secure their apps with only basic security know-how.

    • Consumption-based billing.
    • Auto-scaling.
    • No operations overhead.
    • Global low latency.
    • Secure by default.

    When these benefits are combined with the advantages of GraphQL, one will notice a perfect synergy between the two technologies. Not only does the process become more automatic, the standard facilitates communication between internal teams and makes it easier for clients to integrate with your software, through:

    • API discoverability and automatic documentation.
    • Consolidation of data.
    • Language and data source-agnostic.
    • Type safety.

    Serverless Plus GraphQL Equals Developer Nirvana

    There are numerous examples of companies that have managed to build faster and cut costs with serverless. One of the most impressive demonstrations is when students in Australia managed to rebuild the census application and handle loads that the official $9 million system could not handle, at a fraction of the cost using serverless technology. GraphQL takes these advantages a step further.

    Airbnb, a company already known for its serverless adoption, claims to be moving 10x faster thanks to GraphQL, while Coursera noticed that their frontend developers found it easier to discover and query new data as their API evolves. These benefits are not just limited to a developer’s own company. For example, GitHub clients can now fetch data more efficiently and more easily thanks to the GraphQL-based public-facing API. GraphQL is even competing with SQL—when databases such as FaunaDB offer a native GraphQL API, users move to GraphQL at a rapid rate.

    Given the advantages of serverless in removing server provisioning and maintenance, and the freedom GraphQL gives developers when it comes to using programming languages and database engines, developers are essentially able to build and launch apps on a global scale as quickly as they think of them. And in the end, don’t we all want our customers to have a better experience?

    — Brecht De Rooms

  • The Power of Embedded Analytics and the Build Versus Buy Dilemma

    The Power of Embedded Analytics and the Build Versus Buy Dilemma

    Embedded analytics are a powerful tool for adding value and stickiness to your desktop and mobile apps. Many app developers are providing these features as standard in their tools. But should independent software vendors (ISVs), systems integrators (SIs) or companies building internal apps build analytics tools in-house, or should they buy them from specialist providers?

    There are many considerations to the buy or build dilemma. One reason many developers want to build in-house is the limitation of support services from embedded analytics vendors. Although most providers offer support services, these are not normally included in the price of the package, and are offered at an additional cost. However, in the majority of cases, such support services are in fact effectively a requirement to ensure the visualization is tailored to the use case and look and feel of the app.

    A second consideration is cost. Standard price entry points for embedded analytics can start at $30K to $75K per year. However, behind the upfront pricing structure, there are often multiple levels of service, as well as limits on usage and number of applications the embedded analytics can be used in. This can make pricing far less predictable.

    Creating analytics features in-house may be the best option when working on smaller projects with limited sets of requirements—especially if your development team has a relevant skill set and previous experience developing embedded analytics and data visualizations.

    Lastly, one of the most convincing arguments for building in-house is that product managers remain fully in control over every aspect of their application, not just its functionality but the look and feel as well. By keeping all aspects of development in-house, product teams can control branding, user experience and functionality. The loss of this control is one of the main disadvantages of buying.

    On the other hand, there are many benefits to buying rather than building. The main disadvantage of building in-house is that developers have to switch their focus away from working on the core product to create complex embedded analytics features. Buying saves time and money over training a development team that may lack previous embedded analytics experience and eliminates the need for training where internal resources are simply not available.

    There is also a significant cost associated with building embedded analytics, which on average takes seven months to complete. The estimated average in-house cost can run as high as $350k (based on average U.S. salaries). This includes: four software developers for seven months, one QA professional for seven months, two UX/UI designers for six months and one data scientist for one month.

    Your in-house team is generally responsible for supporting anything built in-house. With the buy option, support is provided by the third party via the cloud and ISVs will not have to allocate resources to fixing issues if and when they occur. As much as 90% of the cost of software during its lifetime is tied to keeping it up and running. Maintenance costs can be significant.

    The time it takes to bring a product to market is another consideration. With average build-it-yourself times taking seven months or more, many product teams decide to buy a bolt-on analytics solution due to the need to release a product as quickly as possible. In a fiercely competitive SaaS market, and with CEOs demanding quick turnaround, buying a pre-built, off-the-shelf solution drastically reduces your time to market.

    Making a final decision on whether to buy or build is a complex task. But in the end, ask yourself:

    • Can my team develop powerful analytics in-house or do I need a specialist? Building analytics tools can be a drain on time and internal resources, and the backend powering even a simple dashboard can be slow and expensive to build from scratch. Plus, you need the resources to maintain it.
    • How does the vendor’s pricing model stack up against my budget? You shouldn’t really be paying any more than what it would cost you to simply build the visualization in-house, so an affordable vendor is essential. However, too often, overly complex pricing structures from vendors result in unpredictable costs, making ROI hard to judge.
    • Has the company received awards for its work? Are they well established and respected in the community? Third-party review sites are helpful here.
    • How much do you trust the vendor? If you are collecting sensitive data, you want to be sure that the vendor isn’t also collecting that information covertly and selling it.

    As more organizations realize just how important understanding their data is, embedded analytics have become the key to harnessing that power. Whether you decide to allocate internal resources or hire a vendor, including powerful embedded analytics and visualizations is virtually a must since it helps you differentiate your app and keep users engaged.

    — Casey McGuigan

  • Three Ways Developers Can Worry Less About Security

    Three Ways Developers Can Worry Less About Security

    Over the past 10 years, developers have risen in organizational importance. They’ve acquired more tools, power, responsibility and autonomy. As part of that evolution, many developers today want to touch all of the layers of code and have their hands in every tool that’s being built. Known as “mechanical sympathy,” developers are taking increased initiative in areas they simply haven’t before–like installing security systems or being in charge of their team’s application security lifecycle.

    While pride of ownership is never a bad thing, in today’s complex IT environments, where organizations are adopting hybrid cloud solutions, microservices and containers–and are required to push applications and software into production as quickly as possible–this simply isn’t scalable. Pursuing mechanical sympathy in this case can be too costly and ultimately become counterproductive–especially when it comes to security, where an attacker only needs to find one vulnerability to gain a foothold in the system.

    Rather than relying on developers to become security experts, organizations should empower them to dig deep on the tasks that matter to them (development) and leverage the expertise of other specialists (security). That way, everyone can focus on their own productivity and deliver the most value to the business.

    Here are three ways developers can let go of their need for mechanical sympathy.

    Consider the Implications of Being Responsible for Security

    Taking on security means abiding by security best practices, like adopting multi-factor authentication and implementing privileged access security for secrets management. None of this is in the typical wheelhouse of the developer, and in fact, developers could actually be increasing risk if they intend on being responsible for security.

    Developers should think of it this way: Imagine something goes wrong with the application you put into production, such as leaking sensitive data based on a mistake in the source code or a design flaw. Do you really want to be the team who’s called? The answer is probably no.

    By focusing on core developer strengths and allowing security teams to handle an organization’s security strategy, developers are able to collectively ship applications that are functionally sound, as well as secure.

    Don’t Solve for Already-Solved Problems

    When it comes to DevOps methodology, there are an increasing number of tools now available that automate time-consuming security tasks–such as automated secrets management, encryption and vaulting solutions–which remove those burdens from developers and allow them to work at true velocity.

    With these tools available, there’s no need to reinvent the wheel. Developers should leverage these assets to their advantage and abstract away complexity to the security team, who should be governing and provisioning security tools so developers can get back to delivering applications to market as efficiently as possible.

    Prioritize the Value Stream

    The aforementioned tools provide a great benefit to developers–freeing them up to focus on more impactful work–but it can be easy for the opposite effect to take hold if developers get mired down in the weeds with them. Instead of trying to specialize in tools such as Kubernetes or OpenShift, which have manufactured obsolescence to begin with, developers should be encouraged to think about the big picture–creating a high-performance, automated pipeline that accelerates software delivery from code to production.

    By focusing on the health of the pipeline, rather than the specific tools within, developers establish a real-time feedback loop that enables them to anticipate the future and see the amazing projects that are truly possible. It positions them as drivers of innovation and increases their sphere of influence across the organization.

    Developers play a critical role in bringing applications and software to market with speed and stability, but when organizations default to letting them own everything from the nuts to bolts, they may end up introducing security issues that could otherwise be avoided.

    The best way to ensure security in the development process, however, is to start with a strong security foundation. Organizations that understand developers and the development process are savvier when it comes to the vulnerabilities that could be inadvertently introduced and are in a better position to secure them with minimal impact on velocity.

    In such a complex and powerful world of cloud, CI/CD and more, when developers are empowered to let go, organizations can better manage and mitigate risk, and deliver higher quality applications that drive true business impact.

    — Brian Kelly

  • Crossing the Ecosystem Boundaries: How ServiceNow, Salesforce and Atlassian Work Together

    Crossing the Ecosystem Boundaries: How ServiceNow, Salesforce and Atlassian Work Together

    Although these platforms were originally designed to serve different functional roles—Atlassian’s help desk, ServiceNow’s IT service management and Salesforces’ sales and CRM—from the outset, they all addressed the evolving mega-management and leadership issue: collaboration across teams and business units.

    Independent action, siloed organizations and disconnected thinking was increasingly giving way to more agile, interdependent, collaborative problem-solving practices and methodologies. Whether the differences were political, cultural, intellectual or institutional, survival became progressively dependent on living, working and communicating more effectively across all boarders. It was in our individual and collective, both internally and externally, best interest to do so.

    Effective companies and the platforms that served them needed to promote a culture that collaborated with less friction across five boundary types (“CCL’s” Center for Creative Leadership’s Boundary-Spanning Leadership)

    • Horizontal: Expertise, function, peers.
    • Vertical: Rank, class, seniority, authority, power.
    • Stakeholder: Partners, constituencies, value chain, communities.
    • Demographic: Gender, generation, nationality, culture, personality, ideology.
    • Geographic: Location, region, markets, distance.

    Collaboration initiatives generally begin by focusing on existing boundaries within a specific business unit or team. Then they expand to forge common ground on specific projects across teams, and finally, moved across all five boundary types with multiple teams and interdependent projects which some refer to as complex adaptive systems. These platforms were launched, quickly developed and continue to evolve to meet this collaboration challenge head on.

    Making Sense of the Blurring Boundaries Across Convergent Platform Roadmaps

    Boundaries are difficult to identify when they are continuously moving and being redefined. Instead of taking a more traditional route of seizing mind and market share by differentiating and establishing/defending clear boundaries, these platforms are doing the opposite, they are converging. The ecosystems that support each platform are also beginning to look increasingly similar—but more on this in a future post.

    The convergence of frameworks such as ITSM and Agile represents a tangential and interconnected set of moving boundaries intersecting somewhere along the spectrum at ITIL—ITIL4 if you’re counting. As all three platforms seek broader market penetration reflecting this full spectrum of frameworks, consider using the ITIL4 Edition’s 17 service management practices as a cheat-sheet for their boundary spanning enterprise roadmaps:

    • Business analysis.
    • Service catalog management.
    • Service design.
    • Service level management.
    • Availability management.
    • Capacity and performance management.
    • Service continuity management.
    • Monitoring and event management.
    • Service desk.
    • Incident management.
    • Service request management–includes both request fulfillment and access management of ITIL V3.
    • Problem management.
    • Release management–includes an explanation about releases in waterfall driven environments and DevOps driven organizations.
    • Change control–formerly known as change management.
    • Service validation and testing.
    • Service configuration management.
    • IT asset management.

    As organizations prioritize and re-prioritize the list reflecting their Agile/ITSM journey, the boundaries between platforms will move, and in many cases, simply disappear. As this journey plays out in your company, the platforms and ecosystems that support each will need be rationalized. Said differently, you may be asked or told to choose one, but as our experience suggests, you won’t need to do that today.

    Instead of diving into the technical features and benefits of each platform and ecosystem, take some time to reflect the benefits of expanding your boundaries to address the business challenges you are facing now and expect over the next 18 months.

    Where does this leave you and your team today?

    While everything else gets worked out….

    • Review the ITIL4 Edition’s 17 service management practices and rank prioritize for your team and company.
    • Do a self-assessment on your personal commitment to, and comfort with, the benefits of expanding your boundaries.
    • Meet with your team and have each of the individuals do the same.
    • Encourage conversations about where the platforms appear to be focusing (e.g., Atlassian focusing more on ITSM and ServiceNow paying more lip service to Agile).
    • When the time comes to make recommendations on app consolidation or choosing one platform over another, your team will be well prepared to do so with open, informed and agile minds.

    One last note on the benefits of expanding boundaries: Leaders who effectively reach beyond their present boundaries have the advantage when faced with solving problems, creating innovative solutions and evolving to thrive in today’s interdependent, complex world. The benefits of boundary spanning leadership include:

    • Increased organizational agility to respond to a dynamic marketplace.
    • Advanced cross-organizational innovation processes.
    • Achievement of mission-critical, bottom-line results:
      • An engaged and empowered workplace at all levels.
      • Flexible, cross-functional learning capabilities to solve problems and adapt to change.
      • Better-managed risks and rewards through enduring cross-sector partnerships.
      • Higher-performing virtual teams.
      • Global mindsets and cross-regional collaboration.

    Use these bullets to get your team spanning conversation started.

    — Joshua Platt

  • Eficode Adds Analytics to Managed DevOps Service

    Eficode Adds Analytics to Managed DevOps Service

    Eficode has added an analytics module to a DevOps platform it makes available as a managed service.

    Tuomas Keränen, product manager for Eficode ROOT, said Eficode ROOT Insights provides different types of end users with access to an analytics module designed to make it easier to understand how changes to a software development project will impact delivery timetables. To make that capability more accessible throughout the organization, Eficode has created dashboards for specific types of end users such as project managers.

    Eficode has crafted a DevOps platform from various tools and platforms to create a toolchain based on best practices it has defined that the company then manages on behalf of end customers. Organizations can choose to have that DevOps platform hosted on-premises or in a public cloud. That approach frees organizations to devote more time and resources to building applications versus managing the DevOps platforms, said Keränen. Eficode ROOT Insights is provided as a complement software-as-a-service (SaaS) application.

    As DevOps continues to evolve, it’s apparent that an alternative approach that relies on external IT service providers to manage the underlying platforms is gaining traction. Many IT organizations simply don’t have the expertise required to build and maintain a DevOps platform. The trade-off is that the best practices are defined by the external service provider, which means the IT organization needs to bend its culture to align with those best practices.

    Another factor that potentially will push organizations toward managed DevOps services is analytics. As the amount of data that needs to be collected to drive analytics applications steadily increases, it becomes apparent that DevOps analytics applications are going to be invoked more easily as SaaS applications. Once an IT organization moves down that path, it’s not too long before the case is made for using the analytics platform to also host the DevOps platform. In addition, as machine learning algorithms and other forms of artificial intelligence (AI) are applied to DevOps, the volume of data required to drive an AI model may require an organization to rely more on external service providers to provide that expertise.

    It’s not at all clear how many organizations may prefer a managed service approach to DevOps. Many of the earlier adopters of DevOps have built custom workflows that are designed specifically to optimize their software development life cycle. However, the adoption of DevOps has been uneven at best. As managed DevOps services proliferate, the number of organizations that can effectively embrace DevOps should increase significantly.

    Of course, managing the tension between internal and external IT service providers is nothing new. However, given the complexity of DevOps processes, relying on a managed service requires a level of collaboration that is not easy to achieve. Just as importantly, once an organization decides to partner with a provider of a managed DevOps service, unraveling that relationship later may prove to be all but impossible.

    — Mike Vizard

  • Mario’s Adventures in Tekton Land

    Mario’s Adventures in Tekton Land

    Tekton is an open source, cloud-native CI/CD pipelines engine. The project has its roots in the Knative project, where about a year ago it started under the name “Knative Pipeline,” with the aim of providing an extension to build Knative’s own “source to image” service. About eight months ago the project was reborn as Tekton and it moved out of the Knative space to its own GitHub organization and it joined the newly formed CD Foundation with a new mission: becoming the common denominator for CI/CD in the cloud-native space. But how is Tekton going to achieve its new aspirations?

    For it to be successful, we wanted Tekton to be intuitive, reliable and easy to operate and maintain. This was discussed in the Tekton working group, and we thought it would be useful if we started using Tekton ourselves as part of our day-to-day jobs, using it to provide CI/CD services to the community.

    Diving into Tekton

    Tekton pipeline aims to provide a set of components used to create a full-fledged cloud-native CI/CD system on top of Kubernetes.

    The core project of Tekton (tektoncd/pipeline) defines a set of Kubernetes custom resources (CRD) as standard constructs for creating CI/CD pipelines. The following is a brief introduction to these CRDs:

    • Task: A sequence of commands (steps) that are run in separate containers in a pod.
    • Pipeline: A collection of tasks executed in a defined order.
    • PipelineResource: Inputs (e.g. git repo) and outputs (e.g. image registry) to a pipeline.
    • TaskRun: Runtime representation of an execution of a task.
    • PipelineRun: Runtime representation of an execution of a pipeline.

    Those pipeline building blocks can be reused, version controlled and curated in a catalog (tektoncd/catalog) that embeds best practices.

    In addition, the Tekton community works on providing more components:

    • Tekton Triggers (tektoncd/triggers) to be able to act upon events (for example: GitHub webhooks, CloudEvents and more).
    • Tekton CLI (tektoncd/cli), which provides a command-line interface focused on the Tekton APIs and features (like getting the logs of your run, starting pipelines and more).
    • Tekton Dashboard (tektoncd/dashboard) which provides a simple dashboard (à-la-kube-dashboard) to view pipelines and logs from your browser.

    Re-Plumbing Test Infrastructure

    We designed Tekton to be intuitive, reliable and easy to operate and maintain. We started using Tekton ourselves as part of our day-to-day job, using it to provide CI/CD services to the community. But we didn’t start from a blank canvas.

    Tekton inherited a few features from the Knative ecosystem, including the CI setup. This was great as it allowed the project to relatively easily kick-start its CI using tools shared with several other projects in the Kubernetes ecosystem (including Kubernetes itself)–a combination of Prow, Tide, Boskos, a few tools from k8s.io/test-infra and several bash scripts contribute to tasks such as running E2E integration test.

    Deployment of the various CI services is manual and the same is true about their configuration. Releases are crafted mostly manually. Since Tekton is made of several projects, we set up tektoncd/plumbing repository to host the shared CI/CD infrastructure: prow configuration, shared test scripts and tools, test docker images. Mario adventure started!

    Mario’s Adventures

    We started to work on what we are calling the Dogfooding project, which is basically using Tekton to build Tekton. And like a ball of wool and a cat, the more we thought about it, the more questions we had. How is Tekton going to integrate with the existing prow-based CI infrastructure? How are we going to test changes to the plumbing repo in isolation? How are we going to deploy the instance of Tekton used to run Tekton CI pipelines? How are we going to do all that with no disruption to our CI service?

    Testing code continuously is essential infrastructure, even more so for cloud-native software, and re-plumbing all at once could leave us without running water. We decided to proceed with baby steps, reducing the area of impact as much as possible and starting our journey tackling the release automation work first.

    Here is an excerpt of the work we did on this journey:

    1. Run our releases using Tekton. That was the initial work, using Tekton to at least release itself, even if it gets triggered manually.
    2. Integrating Tekton and Prow. Prow is able to create Tekton resources since April of this year (#11888 if you are curious), although it took a few iterations to be able to use it on our own CI.
    3. Looking at solutions without using Prow at all. This one was (and still is) the trickiest:
      • We needed something that is able to react based on events (from GitHub), this is where the tektoncd/triggers project comes into place.
      • Prow comes with a UI and tooling to capture and store logs. This piece is missing from the Tetkon project as of today. We do have the tektoncd/dashboard project but it’s early days, there is no support for authentication yet.

    Mario’s Adventures at KubeCon

    Mario’s adventures will continue at KubeCon in San Diego. If you’re attending the conference  and you’re curious to know more, come and listen to our talk, or watch it on the CNCF youtube channel after the conference.

    This article was co-authored by Andrea Frittoli is a developer advocate at IBM and an open source enthusiast. He is a contributor to the Tekton project and he has contributed to OpenStack for several years. He’s a strong advocate for transparency in open source. Andrea has previously been a speaker at FOSSASIA, FOSS Backstage, OpenStack summits, Open Source Summits, Devoxx Days and various meetups.

    — Vincent Demeester

  • How Open Source Software Is Defining the Future of Mobile Connectivity

    How Open Source Software Is Defining the Future of Mobile Connectivity

    Mobile operators face intense pressure to deliver more data, faster connectivity, better coverage and more functionality to end users who are (unfortunately) more demanding than ever. Increasingly, these end users will not be people as we know them but rather Internet of Things (IoT) devices. IDC has projected that by 2025, 60% of the world’s data will be generated by enterprises, double its level in 2017, largely due to the growth in connected devices, sensors, automation and related equipment.

    To address industrial automation, connected cars, smart cities, sensor networks, asset management, connected health and more, operators need to rethink how the connecting network is architected. They must support faster connections, greater density and dramatically reduced latency. And they must add this functionality and flexibility while driving down the costs of deploying, sustaining and managing the network infrastructure.

    For decades, the telecom industry has been dominated by proprietary business and operating models, but with market pressures to find innovative ways forward, operators must find a new approach. That means embracing the open-source principles and innovations that took the computer industry at large from supercomputing to smartwatches and wearables and applying them to wireless network infrastructure.

    Open source software is proving to be key in 5G and IoT development, powering the automation of the mission-critical functions required to support the high speeds and low latency of 5G and the huge number of endpoints in IoT. Stakeholders have recognized that commoditization and democratization of wireless network infrastructure is necessary to stay relevant.

    Several initiatives already are underway to break this proprietary stranglehold and deliver software-defined networking (SDN) into the wireless network. They include the major operators and wireless infrastructure vendors, while disruptive challengers and startups are making an impact, too.

    The operator community along with major enterprises are now actively engaged in collaborative alliances to help drive uptake of open source.

    The O-RAN Alliance includes members such as AT&T, Deutsche Telekom, Intel, Verizon and SK Telecom, and embraces open interfaces and intelligent technologies for the radio access networks (RAN) needed for next-generation mobile applications like augmented reality, IoT, connected cars, smart homes and more.

    The Open vRAN initiative is backed by Cisco and is primarily focused on specifications to enable “unbundling the components of the network into smaller, simpler components and virtualizing capability to reduce cost and increase agility.”

    The Telecom Infrastructure Project’s OpenRAN Group, created in 2016 by Facebook, Intel, Nokia, Deutsche Telekom and SK Telecom, is focusing on more simplified and efficient radio access network technologies based on vendor-neutral hardware and software-defined approaches.

    A key component and enabler of a programmable base station is providing for a software-defined radio, which can revolutionize the traditional RAN landscape and make it easier for operators to roll out tailored services to different customer segments, at a reduced cost, by using software rather than hardware to process radio signals.

    To that end, the MyriadRF open source initiative was founded by Lime Micro in 2012 with the goal of democratizing wireless innovation. It has grown to include contributors ranging from hobbyists and wireless enthusiasts to professional engineers and equipment manufacturers. As of late 2018, more than 8,000 high performance LimeSDR software-defined radio boards had been shipped to developers, with more than 4,000 commits made across 70 GitHub repositories.

    LimeNet’s CrowdCell network-in-a-box platform, which significantly lowers the barrier to entry for software-defined radio development, is opening up a portfolio of new use cases for operators and enables them to go much further than being just a connectivity provider. Furthermore, it has the added benefit of access to a developer community which is constantly creating new applications and modifying and testing existing ones for increased use case fit.

    For example, Vodafone in the UK wanted to extend coverage and add additional services to the network it offers to customers and use this as a means to generate new revenue. By using CrowdCell, Vodafone is able to localize communication to a greater extent and reduce reliance on major cloud service providers–meaning Vodafone gets greater reliability and security at a lower cost.

    They also have the opportunity to develop cloud-hosting revenues and additional services, from which new revenue can enable 5G development and deployment. All this is underpinned by safe access to the app development community that ensures continuous innovation.

    The future of mobile connectivity is software-defined. The software-defined approach and third-party app development will ensure that operators are no longer differentiated solely by coverage or subscription costs but by the value-added services they are able to offer.

    — Tom Canning

  • The NVMF Boogie: How Kubernetes and NVMF Will Rock

    The NVMF Boogie: How Kubernetes and NVMF Will Rock

    I recently wrote a blog about how Kubernetes creates a global computing grid that delivers unprecedented compute flexibility. Then I wrote a blog about the importance of going fully Kubernetes-native. In today’s blog I would like to drill down into the role that the NVMe over Fabrics (NVMe-oF or just NVMF) standard will play in enabling Kubernetes infrastructure flexibility.

    What is NVMe?

    NVMe is a protocol that dictates how a CPU, moves memory via the PCI bus to a storage device. NVMe communicates over a set of rings (per CPU) where commands get submitted from any CPU to the underlying NVMe device. The design of NVMe eliminates intermediate layers between the CPU and the storage device. NVMe devices consist of a controller, queues, namespaces, namespace IDs and the actual storage media with some form of an interface.

    Storage media can be grouped into sections called namespaces with an ID. In the context of NVMF, namespaces provide a way to enforce access control for the disks/consumers. Namespaces are analogous to an OS partition, except the partitioning is done in hardware by the controller and not the OS (you can still have OS partitions on namespaces). Some NVMe namespaces might be hidden from a user (e.g. for security isolation). A controller connects to a port through queues and a namespace through its namespace ID. A controller is allowed to connect to multiple namespaces and a namespace is allowed to be controlled by multiple controllers (and thus also, multiple ports).

    Imagine smearing out this NVMe device across multiple computers and you get to the next important concept, a storage fabric.

    What is NVMF?

    When you put a network between the PCI bus and the storage device, you use NVMe over Fabric (aka NVMe-oF or simply NVMF). NVMF enables fast access between hosts and storage systems over a network. Compared to iSCSI, NVMF has much lower access latency, in practice adding only a small latency difference between local and remote storage. NVMF delivers a breakthrough in throughput and seek time relative to traditional device attached storage:

    Source: Samsung.

    This is possible because NVMF uses RDMA to efficiently access the memory of another computer without using the CPU or operating system of either. RDMA is a transport protocol that bypasses the kernel stack, offloading the workload to network hardware. This means the transport layer takes place in the RDMA NIC itself, so data is transferred directly to or from application memory without having to involve the normal OS-level network stack, achieving low latency with low CPU consumption.

    Of course, NVMF performance is gated by the networking infrastructure used, whether Ethernet, Infiniband or Fibre Channel. The November 2018 final ratification of the NVMe/TCP transport specification has accelerated adoption of NVMF because it allows storage fabrics to be deployed on standard network hardware.

    NVMF can be used with white label storage systems (JBOD), feature-rich block storage systems (SAN), and storage systems built with previous generation (SATA) devices. This means data centers can simultaneously leverage low latency devices while also using more traditional storage subsystems. NVMF also supports using storage class memory as a cache in the data center compute hierarchy.

    Why is NVMF Important?

    Think of NVMe as a replacement for SCSI, and NVMF as a replacement for iSCSI and iSER. NVMF has several distinguishing technical attributes that add up to huge performance scalability at the storage infrastructure layer:

    • Protocol Efficiency: The NVMe-oF protocol is designed specifically for efficient memory transfers at scale. Designed from the ground up for scalability, the NVMe-oF protocol dramatically reduces the internal locking needed to serialize I/O and improves the efficiency of interrupt handling. NVMe-oF also supports MSI-X and interrupt steering, which prevents bottlenecking at the CPU level and enables massive scalability as systems scale. Combined with the large number of cores in modern processors, NVMe-oF delivers massive parallelism to connected devices which increases I/O throughput and lowers I/O latency.
    • Command Set Efficiency: NVMe-oF has a streamlined and simple command set that uses less than half the number of CPU instructions to process an I/O request that SAS or SATA does, providing higher IOPS per CPU instruction cycle and lower I/O latency in the host software stack. NVMe-oF also supports enterprise features such as reservations and client features such as power management, extending the improved efficiency beyond just I/O. NVMEOF devices can approach DDR4 performance levels and, with SCM, become an integral component of the computing memory hierarchy.
    • Queue Parallelism: With 65,535 parallel queues, each with a queue depth of 65,535 entries, NVMe-oF supports massive parallelism. Every CPU has a private queue to communicate with a data device and is capable of achieving high speeds because there are no locks between queues that live on separate cores. Because each controller has its own set of queues per CPU, I/O throughput increases linearly with the number of available cores.
    • Integrated Security: NVMe-oF includes a tunneling protocol that provides security features produced by the Trusted Computing Group (TCG) and other related communities. Features planned for NVMe-oF devices and systems include simple access control, data at rest protection, crypto-erase, purge-level erase and others. Additional information on NVMe-oF security is available in the Trusted Computing Group and NVM Express Joint White Paper: TCG Storage, Opal and NVMe.

    NVMe-oF is designed not only to solve the limitations of scale-out storage, but also to lay the foundation for a storage fabric that dynamically shifts as workloads demand.

    What Makes NVMF Important to Kubernetes?

    Kubernetes applications implicitly need the infrastructure flexibility enabled by NVMF. NVMF and Kubernetes makes distributed computing within sites easier.

    The key point of my Kubernetes-native blog was “(Kubernetes-native) applications break apart into their constituent elements and can deploy and scale across the latency pyramid where they fit best. The elements dance across the infrastructure following their constituents in what I imagine as a containerized ‘ballet’ focused on delivering the best possible user experience at the lowest possible latencies.”

    However, multi-cloud, multi-site and edge deployments introduce significant additional complexity for organizations operating Kubernetes at (or beyond) enterprise scale. Business needs are pushing IT teams to move workloads between sites, clouds, WAN edge and rapidly be able to clone, burst and then destroy workloads anywhere around the world as needed. Yet Kubernetes deployment across multiple regions, sites, clouds and WAN edge remains problematic.

    OpenEBS is the solution for latency sensitive stateful Kubernetes workloads that need universal data agility. A NVMF-based disaggregated storage physical fabric mates perfectly with the disaggregated data agility software infrastructure offered by OpenEBS. NVMF allows data to physically float across a single storage fabric while OpenEBS allows stateful workloads to be loosely coupled across multiple storage infrastructures. And the OpenEBS MayaStor project natively integrates to NVMF to simplify the developer experience.

    Together, NVMF and OpenEBS become the dance floor for those Kubernetes-native application components to do their infrastructure-aware (and infrastructure-intelligent) ballet. Given its performance and scalability, however, perhaps this is better called the NVMF Boogie.

    To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon NA, November 18-21 in San Diego.

    — Charles Piercey