devopsy.kinsta.cloud

Category: DevOps Toolbox

  • DevOps Trends to Watch in 2021

    DevOps Trends to Watch in 2021

    Engineers at fast-paced startups often wear both Dev and DevOps hats. Early in my career, as a developer at a sentiment-analysis startup, I was tasked with writing scripts to spin up servers, configure them, install necessary software and deploy the code we wrote. My custom shell scripts, Python scripts and cron jobs did all the heavy lifting so that we could reliably deploy software and scale resources on AWS. Fast forward to today, and that problem of confidently building and securely deploying software has become an even more enormous challenge at scale.

    The Cloud-Native Application Stack

    A simple looking e-commerce app today can easily be a big set of coordinating microservices that are powered by containers, orchestrated using Kubernetes and hosted on multi-cloud/hybrid cloud. This combination enables faster delivery of code, cheaper hosting bills and more efficient use of server resources.

    While the application components certainly became smaller over time, organizations today are tasked with managing tens of thousands of containers in a production environment. In a 2020 Cloud Native Computing Foundation (CNCF) survey, 92% of respondents said they use containers in production, a 300% increase from just 23% in the first survey from March 2016. Such coordination complexity at scale is being managed by software delivery pipelines. Just as Ford revolutionized the automobile industry with its assembly line, these integrated DevOps products are massively impacting verticals like finance and insurance.

    DevOps Trends

    Trend: Developers are now Decision Makers and Influencers

    We are excited to see the increasing impact and influence of developers when it comes to the selection of tools. The top-down sales motion largely doesn’t work in the cloud-native world, and needs to be supplemented by bottom-up community adoption. The best developers pick peer-recommended tools to solve these tech challenges. Developers today leverage large communities, self-organize and pick the best open source tools to kickstart small projects that have potential to impact the growth curve of any organization. For vendors, developer evangelism becomes an important strategy for customer acquisition.

    Trend: Integrated Software Delivery Pipelines

    CIOs typically pick category-leading products for each category of the DevOps toolchain, like CI, CD, SAST and SCA. Historically speaking, integrating these tools to maintain an effective CI/CD pipeline has been painful and requires stitching together tools using custom scripts. We hear the frustration when we speak with maintainers of such scripts. To address this challenge, we think end-to-end integrated DevOps platforms will become the norm. Such platforms will offer a few best-in-class components with built-in automation, and will enable organizations to integrate other DevOps tools in a plug-and-play manner.

    Trend: Shifting Left with Code

    The shift left phenomena, which enables fixing of bugs/security issues early in the software development life cycle when it is cheapest to do so, is accelerating. By implementing a shared security responsibility model and enabling developers to take more ownership related to infrastructure, security and other critical production issues with code, organizations are shipping code faster and cheaper with less bugs.

    Trend: Low-Code DevOps Automation

    Many workflows are pretty standard in the DevOps world. Go look at an alert in production; prioritize it, look at the knowledge base for a fix, create incident, apply fix, resolve ticket and then repeat. This example is an oversimplification, but there are many repeat use cases that can be automatically managed. These workflows can and should be automated, which can make DevOps teams more effective. We think that customizable low-code automation interfaces will evolve to act as secret weapons for effective DevOps engineers.

    Trend: Security at the Speed of Dev

    Software release cycles a decade ago used to be four to six months long — if not longer — but today, the software delivery life cycle and release cadence is accelerating. The 2019 State of DevOps report suggests that the top performers deploy software 1,460 times per year, which is effectively four deployments per day. Security, compliance, vulnerability management, authentication and authorization all have to move at this speed, otherwise delivery of secure code is just not possible. We see the most innovative CISOs adopt a shared ownership model of security that enables application component owners to detect and fix vulnerabilities. In this shared model, a good balance between agility (so that development is not blocked) and security (so that risk can be managed) needs to be struck.

    Are You Building the Next Big Thing?

    Cloud infrastructure is an enormous opportunity that Gartner expects to grow to $412 billion by 2024 and many enterprises are just getting started. If you haven’t been paying attention, Satya Nadella now runs Microsoft and Andy Jassy will soon run Amazon. Both these leaders are pioneers in cloud, having led Azure at Microsoft and AWS at Amazon, respectively. This shows how much the largest companies, as measured by global market cap, believe in the opportunity related to the cloud. We couldn’t agree more.

  • Redis Labs Extends Reach of In-Memory Database

    Redis Labs Extends Reach of In-Memory Database

    At its online RedisConf 2021 event, Redis Labs today detailed how the company’s in-memory database will be employed to drive real-time applications that are at the core of most digital business transformation initiatives.

    Yiftach Shoolman, Redis Labs CTO and co-founder, said there are four initiatives at the core of the Redis Labs effort to replace legacy databases that organizations have historically employed to drive batch-oriented application processes.

    RedisRaft, which will become available with release 7.0 of the Redis database in the third quarter, provides a module based on the Raft consensus algorithm that enable multiple Redis servers to be consumed by an application as if they were one single, fault-tolerant, logical cluster. That capability will provide IT teams with the ability to distribute Redis servers closer to the point where data is being consumed by a distributed application without incurring any conflicts between databases, said Shoolman.

    At the same time, Redis is extending its Active-Active technology to enable integrated data models to be deployed in a way that allows distributed applications to run as close as possible to the point where data is being created and consumed.

    Finally, RediSearch is being updated to include support for indexing capabilities in JSON applications using a nested document model, while a RedisAI capability will add an inference engine for artificial intelligence (AI) applications that can be employed as a feature store through which the deployment of AI models within applications can be managed. That approach also promises to significantly increase the overall performance of AI applications, noted Shoolman.

    Historically, the Redis in-memory database has been best known for the caching capabilities it provides developers. However, Shoolman noted 66% of organizations that employ the Redis database are using it for additional use cases. As microservices that are often configured with their own database become more prevalent, the adoption of an in-memory database platform should only increase, added Shoolman.

    Many of those microservices are being employed within the context of a digital business transformation project. The issue that organizations are coming to terms with is most of those processes need to run in near real-time. That requirement will eventually push more organizations toward deploying a database based on a distributed database running in-memory that appears as one logical entity to an application, said Shoolman.

    At its core, Redis a data store that supports structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes, and streams. Among other capabilities, it includes support for built-in replication, Lua scripting, transactions and varying levels of persistence. The company claims the core Redis datastore has been downloaded 1.5 million times in the last nine years.

    On top of that installed base, Redis Labs then makes available tools for deploying and managing Redis within large-scale application environments. Most recently, Redis Labs raised an additional $100 million round of financing to bring its valuation to above $2 billion. The company now claims to have more than 8,000 paying customers, with a 54% compound annual growth rate in sales for the past three years.

    It’s hard to say to what degree Redis data stores might supplant legacy database platforms. However, the one thing that is certain is there are lot more databases running in memory that, in many cases, obviate the need for a traditional relational database.

  • DevSecOps Practices Gap Assessment

    DevSecOps Practices Gap Assessment

    DevSecOps practices are essential to deliver software safely and securely withinDevOps value streams.

    To get the maximum security protection from DevSecOps, it is important to use recommended best practices, inclusive of people, processes and technologies. A gap assessment is a great way to efficiently evaluate an organization’s practices for DevSecOps and determine a strategy for improvement. Gap analysis provides valuable inputs for formulating a strategy and roadmap to improve a topic including DevSecOps.

    My SlideShare Gap Survey, Assessment and Analysis For DevSecOps explains how my 9 Pillars Gap Assessment Tool greatly speeds up the gap survey, gap assessment and gap analysis process for DevSecOps.

    It is important to clarify a few definitions for these purposes, because I have found that different people and organizations use these terms differently. In this blog, and in my consulting work, I use the following definitions:

    • Gap survey is discovery tool used to collect information about the current state of practices for a topic.
    • Gap assessment is a process to determine differences (gaps) between current practices and recommended practices for a topic. Gap assessment is also used to refer to the entire collection of activities in this list.
    • Gap assessment workshop is a meeting in which the gap assessment is validated prior to gap analysis.
    • Gap analysis is the process of determining priorities for reducing the gaps found during a gap assessment.
    • Gap assessment tool is software that helps perform gap surveys, gap assessments and gap analysis.

    For those that want to dig deeper on this topic, additional definitions and contextual information related to the way I do gap assessments are clarified in my book Engineering DevOps.

    I define a seven step process for gap assessments. The steps are as follows:

    1. Pick a topic to analyze. The same processes and tools can be applied equally well to almost any topic.
    2. Determine recommended practices for the topic. Something that is worth doing a gap assessment for, such as DevSecOps, will normally be fairly complex with tens, if not hundreds, of practices. Categorize the practices into groupings (I call the groups “pillars”) to make the list of practices more understandable and meaningful. Doing this will require some research and work. For example, I have a database of practices and pillars for DevOps, continuous testing, DevSecOps and SRE and some other topics such as containers.
    3. Enter the chosen practices into a gap assessment tool. The tool can be a simple survey tool you create, a commercial survey tool or something custom-made.
    4. Determine who needs to be included in the gap survey.
    5. Collect data using the gap survey.
    6. Perform a gap assessment.
    7. Perform a gap analysis.

    Let’s take a look at the above seven steps for DevSecOps.

    Step 1: Pick a Topic to Analyze

    For our purposes here, the topic is DevSecOps.

    Step 2: Determine Recommended Practices for DevSecOps

    For an example DevSecOps assessment, I used the practices under nine practices categories described in a blog that I co-authored called 9 Pillars of Continuous Security Best Practices. The nine practice category pillars are:

    • Leadership
    • Collaborative culture
    • Design for DevOps
    • Continuous integration
    • Continuous testing
    • Continuous monitoring
    • Elastic infrastructure
    • Continuous delivery
    • Continuous security

    Step 3: Enter the Selected Practices into a Gap Assessment Tool

    You can find and download a free gap assessment tool, pre-loaded with a sample of DevSecOps practices in a file called DevSecOps Assessment, from one of the resource pages found on my website, EngineeringDevOps. You can edit the practices categories and add/delete practices from each category if you prefer to make changes.

    Step 4: Determine Who to Include in the Gap Survey

    For the gap analysis to be comprehensive, people in roles that are affected by container practices need to be surveyed, or at least represented, to ensure their perspectives are included. Some example roles that are typically included:

    • Business leaders – because leaders influence culture and budgets for DevSecOps.
    • Developers – because developers need to design in accordance with DevSecOps practices.
    • Project owners – because they influence product work priorities.
    • QA testers – because they need to test in accordance with DevSecOps practices.
    • Ops – because DevSecOps practices affect operations.
    • Security – because DevSecOps practices affect security practices.

    The survey can be conducted for an individual application, a group of applications or all the applications in the enterprise. However, it is important that the gap assessment and gap analysis be performed on the organizational segment that is being targeted for improvement.

    Step 5: Collect Data Using the Gap Survey

    A gap survey should allow each surveyed person to enter an “Importance Level” score, a “Practice Level” score and comments for each practice. All this information is essential for the gap assessment and gap analysis. In the gap survey included in in my gap assessment tool, survey respondents are asked to score the importance of each practice as one of 0=not relevant, 1=not important, 2=nice to have, 3=important, 4=very important or 5=critical. Practice-level choices are 0=not sure, 1=rarely, if ever, 2= sometimes, 3=most of the time, 4=always or 5=we are good at this. Comments that are relevant to qualify or explain the scores entered for each practice should also be entered, especially if there is any doubt or ambiguity.

    Step 6: Perform a Gap Assessment

    The gap assessment process requires all the practices scores collected from the surveys to be collected and assembled to calculate an aggregate set of scores. Gap scores are calculated using a formula that weights each practice level score with the corresponding importance level score. A visual representation helps to identify practices areas and individual practices that have the highest, most important gaps.

    No matter how professionally written the practices and score definitions are, it is not unusual for some people to misunderstand and to enter scores that they would not have otherwise. For this reason, it is important to ensure that the data collected is validated before conducting the gap analysis. The preferred approach is to conduct a gap assessment workshop with key representatives from each role that participated in the survey. During the workshop, the scores for each practice that have a high deviation between survey responses are discussed and, if necessary, adjusted.

    Step 7: Perform a Gap Analysis

    The gap analysis process involves extracting the high gap practices and tagging and ranking each of them against solution categories that are determined by a consultant or topic expert.

    The result of the gap analysis indicates where solution strategies and implementation roadmaps need to be focused to reduce the most important gaps.

    What This Means

    While is it clear that DevOps offers immense value for software deployment, the adherence to best practices for DevSecOps is essential to reduce risk and assure security. Each organization is different, and has different security postures. The specific practices, security tools and metrics that are appropriate may vary according to a risk assessment for each organization. This blog explained a gap assessment approach, which uses my nine pillars of best practices for DevSecOps. My free DevSecOps assessment tool facilitates a gap analysis for DevSecOps and priorities to improve DevSecOps practices for an organization.

  • Unified Enterprise DevOps Platforms Eliminate Bottlenecks

    Unified Enterprise DevOps Platforms Eliminate Bottlenecks

    Nearly all enterprise digital transformation projects include moving software to DevOps processes and platforms. The effort, cost and time it takes to be successful with these transformations depend on a lot of factors, but one thing stands out: as indicated in my white paper Unified Enterprise DevOps Platform Why? How? And What?, “An enterprise with many variations of DevOps platforms and tools is at best chaotic, and at worst, is a gross impediment to overall organization agility – undermining the very purpose that DevOps is meant to serve.”

    In my recent blog Why a Unified Enterprise DevOps Platform is Necessary, I indicated that “The aggregate set of platforms and tools is like a Frankenstein’s monster, with many incompatible parts resulting in an awkward looking product that is hard to control, replicate and evolve.”

    Success with DevOps requires frictionless interplay between the people, processes and technologies that need to cooperate for continuous deliveries to happen effectively. The existence of different DevOps solutions creates organizational barriers to unification, because different groups become dependent on their local variations.

    A Unified Enterprise DevOps Platform Accelerates Time-to-Value

    Culture barriers, process problems and DevOps platform “wars” all are factors impacting the progress of digital transformations. A unified enterprise DevOps platform would accelerate time-to-value, and realize efficient, safe operation for digitally transforming enterprises and organizations.

    The following are general requirements for a unified enterprise DevOps platform.

    ● A new, unified approach that supports end-to-end capabilities for continuous delivery of software, as a managed service, for multiple streams of pipelines.
    ● Unified metrics that show business value, costs of the value stream and timing for end-to-end continuous delivery flow across multiple pipelines.
    ● Unified capabilities including visibility, orchestration, integration, security, governance, traceability and management of flow for application value streams across multiple pipelines.
    ● Capabilities to unify continuous testing/QA, troubleshooting and debugging.
    ● A new, unified approach to simplify and accelerate toolchains that support new cloud-native and cloud-adapted applications, infrastructures, machine learning, commercial off-the-shelf and open source software.

    A DevOps Platform Blueprint

    In my SlideShare DevOps-as-a-Service Values, I described a four-layer blueprint for a unified enterprise DevOps platform that satisfies these general requirements. The main elements in the blueprint are:

    ● Stakeholders – There are three types of stakeholders: business, appdev users and DevOps admin teams.
    ● Service access layer – Consists of portals and APIs for business, appdev and DevOps administration users and access security.
    ● Service layer – consists of user services and DevOps admin services.

      • User Services include things that appdev users need to perform their application development and testing work. This includes capabilities for no-code automation and orchestration, repos for code, artifacts and data, multi-CI/CD pipelines and sandboxes, test data, configuration, continuous testing/QA, service level objectives (SLOs) for business and appdev users and troubleshooting and debugging.
      • DevOps admin services include things that a DevOps team needs to administer and manage the platform. This includes capabilities for platform administration, the ability to import pipelines into the unified platform, integration with enterprise service catalogs, integration with enterprise issues and change request systems, SLOs for DevOps administrators and capabilities to implement governance policies-as-code.

    ● Resources layer – includes resources that the services need to run including application stacks, infrastructure resources, tools, infrastructure code and topology data.

    One of the most important benefits of a unified platform is the ability to unify SLOs for each class of stakeholders. The SLOs need to be chosen to best fit the requirements of each application.

    Business value can be demonstrated by the following SLOs:

    ● Productivity metrics.
    ● ROI data tracking.
    ● Agility of services – Lead time for changes and frequency of releases.
    ● Reliability of services – Change failure rate; time to restore services.
    ● Governance compliance and security.

    Appdev user value can be demonstrated by the following SLOs:

    ● Platform availability.
    ● Service response time.
    ● User experience scores.
    ● Change request response time.

    DaaS team value can be demonstrated by the following SLOs:

    ● Scope of applications – number of apps supported, number of users per month.
    ● Number of requests served per month.

    Without a unified enterprise DevOps platform, all these SLOs would have to be implemented multiple times, once for each type of DevOps platform.

    To realize DevOps-as-a-Service for enterprises with multiple related pipelines, each pipeline needs to be configured to work within a unified model where common tools process each application and microservice continuous delivery task in parallel. Unified control and monitor points are needed – and should be exposed by the unified platform – to realize SLOs in a unified way. Without a unified platform, all of these SLOs will need to be implemented separately, for each platform, and then another system will be needed to aggregate them into a common view.

    The SLOs will indicate where the biggest bottlenecks are for a particular pipeline. In many enterprise pipelines, the biggest bottleneck is testing, troubleshooting test failures and debugging to resolve the failures. It is a myth that continuous testing is just the execution of automated tests. The whole range of “testing things” includes test planning, test resource scheduling, test design, test implementation, test environment management, test execution, test results reporting, test result analysis, troubleshooting, debugging, fix retest and test management, and all can be sources of bottlenecks. These “testing things” are all sources of bottlenecks for every stage of a DevOps pipeline, for all Three Ways of DevOps. Without a Unified enterprise DevOps platform, all these testing things must be addressed separately, in accordance with the different pipeline platforms.

    Testing, troubleshooting and debugging tasks are a major source of bottlenecks for most DevOps value streams. A unified platform improves continuous testing/QA, troubleshooting and debugging as follows:

    ● Continuous testing/QA – Orchestration of test environments, execution of tests and reporting test verdicts is unified.
    ● Troubleshooting – Failures are displayed in a consistent format, including trends correlated to pipeline and application log data, to simplify analysis of test failures.
    ● Debugging – Causes of failure can be isolated with intelligent analysis of troubleshooting results, and the platform can quickly install the fixes and orchestrate the test environment to verify the fix is working.

    These testing, troubleshooting and debugging advantages have a major impact on the ability to accelerate digital transformations.

    What This Means

    Digital transformation of many enterprises and organizations are being seriously impeded by having too many DevOps platforms and tools. A unified enterprise DevOps platform resolves the technical problems of broken, leaky, “Frankenstein” toolchains and processes. Enlightened leaders that understand the importance and the broad impacts of unified enterprise DevOps platform are wise to recognize the sense of urgency surrounding a migration to a unified platform, and take action to stop the proliferation of too many platforms.

    To realize a unified enterprise DevOps platform in an enterprise takes more than simply buying and installing a new platform and telling everyone to start using it in place of what they are used to. With multiple entrenched DevOps platforms in place, there are practical and political considerations that must be overcome. Introducing any changes across a large group of applications in an enterprise can be chaotic, and will often be more costly and take too much time if the changes are not well-coordinated in accordance with a carefully crafted strategy and implementation roadmap that leaders and team members are aligned with.

  • DevOps Skills Increasing Among Salesforce Developers

    DevOps Skills Increasing Among Salesforce Developers

    A survey of 521 attendees at a recent online Future of Salesforce DevOps Summit event hosted by Gearset, a provider of a continuous integration/continuous delivery (CI/CD) platform for Salesforce developers, found just under half (47%) of respondents now release software at least multiple times a week, with another 29% releasing software weekly or biweekly.

    More than half of respondents (52%) also noted they now can deploy software in less than an hour.

    Gearset CEO Kevin Boyle said the survey suggested the level of DevOps maturity among developers building applications on top of the Salesforce software-as-a-service (SaaS) platform is increasing rapidly. In the wake of the COVID-19 pandemic, many organizations embraced SaaS platforms to accelerate their digital business transformation initiatives rather than building entire applications from scratch.

    However, the biggest benefit cited by survey respondents was not the ability to release applications faster, but rather the improvements to application quality (77%), followed by increased productivity (69%) and enhanced collaboration (46%). Other benefits cited included reduced complexity thanks to automation; increased agility; more reliable releases and time and cost savings.

    Overall, the survey identifies source code control, CI/CD, unit testing, change monitoring and backups of metadata and data as being the most widely implemented DevOps capabilities that have been adopted. Over the next 12 months, capabilities that will see the most increased adoption among survey respondents include CI/CD, static code analysis and sandbox seeding with data.

    Survey respondents said their top obstacles to adoption of DevOps are ramp-up time, followed by difficulties adopting source control and a perceived steep learning curve. For many organizations that have adopted Salesforce, the intimidation factor when it comes to adopting DevOps best practices is still fairly high, Boyle noted.

    Most recently, Gearset added a free training platform, dubbed DevOps Launchpad, that employs gamification techniques to make DevOps more accessible to developers. Topics covered include version control fundamentals, Git branching strategies and Salesforce backups.

    Gearset counts McKesson, Zillow, Traction on Demand, Intercom, Accenture, IBM and Johnson & Johnson among its customers. It’s not clear to what degree appreciation for DevOps has risen among smaller companies. However, as the number of custom applications deployed as extensions to the Salesforce platform increases, it is apparent that manual processes for building and deploying applications will not scale. The Salesforce platform itself is uniquely constructed using a set of objects and tables that generally lead organizations to adopt a DevOps platform built for that environment. However, it is possible to build and deploy applications using a DevOps platform designed for building and deploying applications that might need to span multiple platforms. Those applications would be integrated with the Salesforce platform at an application programming interface (API) level.

    Regardless of how custom applications are built and deployed, it’s clear Salesforce platforms are becoming more widely used in enterprise IT environments. Each DevOps team will need to determine how best to integrate those platforms within the context of their overall DevOps strategy.

  • BMC Tightens Integration Across DevOps Tools for Mainframes

    BMC Tightens Integration Across DevOps Tools for Mainframes

    BMC has extended its BMC Automated Mainframe Intelligence (AMI) and BMC Compuware portfolios as part of an ongoing effort to make mainframes just another platform for DevOps teams to build and deploy applications on.

    John McKenny, senior vice president and general manager for Intelligent Z optimization and transformation at BMC, said the DevOps technologies that the company gained last year with the acquisition of Compuware have now been fully integrated with the rest of the BMC mainframe portfolio.

    The BMC Compuware continuous integration/continuous delivery (CI/CD) platform has been integrated with Visual Studio Code, a free tool from Microsoft that is employed widely by developers, he added.

    The BMC Compuware ISPW solution integrates with Visual Studio Code to enable developers to compile changed code on the mainframe with a right-click.

    In addition, the BMC Compuware Topaz platform for applying DevOps practices to the development of mainframe applications has a new command-line interface (CLI) along with support for multiple application programming interfaces.

    A BMC Compuware File-AID 21 tool now makes it possible to reduce the time required to search billions of transactions and log files on a mainframe, while the BMC AMI Security tool has been integrated with the BMC Helix IT service management (ITSM) platform.

    Finally, the BMC AMI Ops Insight tool has been updated to provide diagnostic capabilities to more easily identify the probable cause of an issue, while the BMC AMI SQL Performance for DB2 tool now includes a plug-in for the open source Jenkins continuous integration/continuous delivery (CI/CD) platform.

    McKenny said the goal is to integrate mainframes with any DevOps workflow as part of an effort to either deploy new applications or modernize legacy applications already running on the platform. Modernization of mainframe applications is being driven by the need to add application programming interfaces (APIs) to mainframe applications that need to be integrated with a much broader array of distributed applications, he noted.

    Much of the demand for those APIs is being driven by digital business transformation initiatives that require applications to be able to access backend transaction processing and analytics applications running on mainframes, added McKenny.

    It’s not clear at what rate IT teams that build and deploy applications on mainframes are moving away from waterfall-based processes to embrace best DevOps practices. However, McKenny noted, the rate of changes being made to mainframe applications has accelerated significantly over the last few years.

    While large numbers of applications have been migrated off mainframes over the last decade, the number of enterprise IT organizations that continue to rely on the venerable platform to process transactions remains considerable. Many organizations aren’t convinced that it’s worth rewriting the applications they have relied on for decades when it’s easier to make them more accessible to other applications by adding an API. In effect, McKenny noted, mainframe applications are becoming just another microservice that can be invoked by a developer.

    Regardless of the motivation, it’s clear mainframe applications will continue to run well into the next decade. The challenge IT teams now face is finding a way to make those mainframes just another platform on which to deploy applications at the end of a DevOps toolchain.

  • Languages and DevOps: Can’t be Contained

    Languages and DevOps: Can’t be Contained

    The evolution of platform is an interesting study. From physical to virtual to cloud to containers, the progression has been steady, and each successive platform has benefits for both Dev and Ops that make them appealing. In the case of containers, portability is the word.

    As easy to fold into your DevOps toolchain as cloud, yet able to be deployed wherever you need, containers have absolutely taken the industry by storm. While all cloud vendors offer container options these days, their “value add” capabilities are thinly veiled attempts at vendor lock-in, so most organizations have avoided them.

    Development languages are as diverse as they are for physical or cloud; you can pretty much find containers hosting every language imaginable. That makes perfect sense, since one use of containers is to keep an otherwise-obsolete environment alive. That’s certainly not the primary use of containers, but it is an important one. The catch with containers is that the primary languages depend greatly on usage. A container that is persistent and intended to last months or even years will tend toward a different set of languages than one that is ephemeral and intended to last only minutes.

    The big weaknesses of containers – security, persistent storage, and management – have been addressed more and more as container usage has grown. Complexity of environments is now the big issue, but that is not inherent to containers; merely enabled by the ease with which new machines can be spun up.

    For longer-term containers, Java and C# are still the languages of choice, though others are certainly making headway. A combination of how new container technology is, and the extra layer between applications and hardware in the current popular container solutions make C/C++ less popular than on physical machines. They are still used because like other targets, containers still have operating systems that are linked to more easily in these languages.

    Shorter-lived containers tend to use languages that grew up in the Agile environment – from Python to JavaScript to Go. The ability to quickly crank out the code and get to test is useful for Agile and DevOps environments.

    Hooking into DevOps toolchains is at least as easy as it is in a cloud environment. There are even a fairly large number of companies that have moved to spinning up a container to build, then destroying it, only to do it all again on the next build. This is not mainstream yet, but the ability to completely configure the container and its loaded toolset makes it an appealing option.

    The kicker – one that cannot be overstated – is that it is simple and easy to get developers running with a copy of a complex system that once would have required several physical boxes. I, personally, have worked on a seven-plus server client system that I had a complete copy of on my (admittedly beefy) development laptop. It wasn’t the most performant, but it was enough to do unit testing.

    The security weaknesses of cloud come along with containers too – there is a lot more than just configuring an IP to directing traffic, for example – but mitigating tools and processes are available.

    This pundit truly believes that containers are the future of DevOps. They have the best of previous iterations, while offering mobility that just didn’t exist in other solutions. The ability to move an entire infrastructure from one deployment target to another (data center to cloud, for example), or from one cloud vendor to another makes this option the king for organizations that require (for business objectives, compliance or finance reasons) the ability to move their applications quickly.

    Meanwhile, rock on.

    Containers are the cutting edge of wherever it is we are headed. They are more agile than any other solution, they do not limit programming or networking options, can be run wherever the organization needs them and can be modified or redeployed easily. There is more complexity, but most of it has been obfuscated; only when things go terribly wrong is the complexity really an issue. Development for containers, with a few exceptions, is development for the target OS. That makes it easy to keep the organization’s systems preferences while enabling developers. All good. And many (I daresay most) of you are already taking advantage of them. Keep rocking it. Use what the org needs, and don’t forget to check every once in a while to make certain what is in use is what needs to be in use.

  • Linux Foundation Takes Control Of AsyncAPI Specification

    Linux Foundation Takes Control Of AsyncAPI Specification

    The Linux Foundation announced today that it now plays host to an AsyncAPI Initiative that has developed a specification and a suite of open source tools for asynchronous application programming interfaces (APIs) and event-driven architectures.

    Founding sponsors of the AsyncAPI Initiative include Ably Realtime, Apideck, Bump, IQVIA Technologies, Slack, Solace and TIBCO. Most recently, Postman, a provider of API management tools, committed to providing engineering resources to the project as well.

    AsyncAPI provides a language for describing the interface of event-driven systems, regardless of the underlying technology, and supports the full development cycle of event-driven architecture. That approach helps unify documentation automation and code generation, as well as managing, testing and monitoring asynchronous APIs.

    Chris Aniszczyk, CTO and vice president for developer relations at the Linux Foundation, said the Linux Foundation will work with the member organizations to facilitate a governing framework similar to the frameworks other open source projects have adopted under its auspices.

    In general, reliance on event-driven architectures is on the rise thanks, in part, to the rise of digital business process initiatives that require the processing and sharing of data across multiple platforms in near-real-time. The AsyncAPI Initiative makes it simpler for disparate event-driven platforms to be integrated. It’s unlikely multiple organizations involved in a digital business initiative are ever going to standardize on the same event-driven platform, so an AsyncAPI specification provides a critical linchpin.

    Usage of the AsyncAPI Initiative is already fairly widespread, with Adidas, PayPal, Salesforce, SAP and Slack all employing it in production environments. In fact, the Linux Foundation cited research that identified AsyncAPI as the fastest-growing specification being implemented by developers, after seeing adoption rates in production environments triple from 2019 to 2020.

    Event-driven IT architectures have, of course, been around for decades. However, as they have gained traction in the modern era of software development, the need for the equivalent of a REST API for asynchronous communications between platforms, based on different event-driven architectures, became apparent. The AsyncAPI doesn’t replace the need for other types of API specifications, but Aniszczyk noted that, over time, it will soon player a larger role, especially as cloud-native application environments become more distributed and the number of use cases that can’t be addressed using traditional batch-oriented techniques continues to rise.

    In the meantime, the number of providers of event-driven platforms based on proprietary architectures spans vendors ranging from Oracle and IBM to cloud service providers such as Amazon Web Services (AWS). At the same time, open source messaging platforms such as Apache Kafka are starting to have a significant impact on how IT teams build their own event-driven platforms.

    The level of enthusiasm for an API specification that makes all those platforms more interoperable than they would otherwise be is unclear. However, as modern IT platforms continue to evolve, more IT teams are becoming aware of the crucial role API specifications play in preventing them from becoming locked into one platform.

  • CircleCI Previews Support for Arm Platforms

    CircleCI Previews Support for Arm Platforms

    CircleCI today previewed the ability to deploy applications on platforms based on Arm processors in a forthcoming release of its namesake continuous integration/continuous delivery (CI/CD) platform.

    Tom Trahan, vice president of business development for CircleCI, said that Arm processors are rapidly being embraced by DevOps teams seeking to take advantage of the price/performance advantages they currently have over conventional x86 processors. Amazon Web Services (AWS), for example, makes available an AWS Graviton2 service based on 64-bit Arm Neoverse processors that it claims provides a 40% advantage in overall price/performance.

    The Arm community, meanwhile, has adopted many of the libraries employed on x86 platforms, Trahan noted, which is making it simpler to automate the deployment of applications on Arm-based platforms using a CI/CD platform.

    It’s not clear how quickly developers are building applications for Arm processors. Historically, Arm processors can be found in a wide range of embedded systems typically deployed at the network edge. However, cloud services based on Arm processors are also becoming more widely available, and it’s only a matter of time before more Arm-based platforms are deployed in on-premises IT environments.

    Arm itself is also in the process of being acquired by Nvidia, which envisions running machine learning algorithms, and other classed of advanced workloads, on both Arm processors and graphics processing units (GPUs). Many of the early efforts to build processors optimized for running AI workloads at the network edge are based on Arm processors.

    The addition of Arm platforms to any IT environment will naturally create some management challenges for DevOps teams. In some cases, developers may even decide to build applications using, for example, the latest generation of machines from Apple, that are based on Arm processors, to streamline the application development and deployment process. In other cases, however, applications that are intended to be deployed on Arm processors might still be built on x86 systems.

    Regardless of approach, the arrival of more Arm-based platforms in the enterprise doesn’t mean other classes of processors are going to fade away. Rather, from a processor perspective, IT environments are about to become a lot more diverse as more workloads are deployed on a wider range of processors. It may be quite a while before Arm-based platforms usurp x86-based systems that still dominate the IT landscape. However, during any turbulent economic time, the number of organizations inclined to reduce the total cost of IT by embracing new platforms always increases.

    Fortunately, as more IT infrastructure is managed as code, the easier it becomes to switch platforms. There may even come a day when the decisions about which platform to employ and when is completely automated by machine learning algorithms trained to maximize IT infrastructure efficiency to the fullest extent possible.

    In the meantime, DevOps teams should expect, to one degree or another, to revisit their workflows processes in the months ahead. Not only are more workloads being pushed out to edge computing platforms where Arm processors are commonly found, those Arm processors will soon find their way into the rest of the extended enterprise.

  • Tricentis Aquires Neotys to Gain Load Testing Tools

    Tricentis Aquires Neotys to Gain Load Testing Tools

    Tricentis announced today it has acquired Neotys as part of an effort to expand the scope of its tools portfolio into the realm of load testing. Terms of the deal were not disclosed.

    Neotys provides application testing teams with automatic test design and maintenance, user behavior simulation and root cause analysis capabilities using a set of tools that are designed to be integrated within a DevOps toolchain.

    Sandeep Johri, Tricentis CEO, said the Neotys platform will extend the scope of Tricentis’ Tosca testing platform that today enables DevOps teams to employ machine learning algorithms to test the functions within an application.

    In general, Johri said there’s now a lot more focus on load testing, thanks in part to the rise of microservices-based applications that are starting to be deployed at scale. DevOps teams are, first, trying to identify, as early as possible, a single point of failure that might bring down an application. Second, modern microservices-based applications are designed to degrade gracefully when there is no single point of failure. Identifying what issues are leading to that degradation can be challenging, given all the dependencies that might exist within a microservices-based application.

    Neotys claims to have more than 600 enterprise customers, including BNP Paribas, Dell, Lufthansa, McKesson and Verizon. Tricentis claims to have more than 1,800 customers, including McKesson, Accenture, Nationwide Insurance, Allianz, Telstra, Rockwell Automation, Moët Hennessy-Louis Vuitton (LVMH PCIS) and Vodafone. The acquisition of Neotys will reduce the number of vendors that an organization needs to engage to gain access to a portfolio of testing tools, Johri noted.

    As IT organizations embrace DevOps best practices, more responsibility for application testing is clearly shifting left toward developers. However, it’s not quite clear to what degree that shift is either eliminating or reducing the need for dedicated application testing teams at the back end of the application development process.

    At the same time, many rote testing tasks are now being automated using machine learning algorithms. Those algorithms don’t completely eliminate the need for humans to test applications, but they do significantly reduce the testing time required. They should also create more time for developers to collaborate with one another to review more complex interactions within an application that machine learning algorithms are unlikely to be able to automate any time soon.

    The amount of time organizations devote to application testing has always been constrained. As deployment deadlines approach, often the first thing that gets reduced is the amount of time allocated to application testing. In many cases, developers deliberately don’t address some known issues until the first or second update to an application that has already been deployed in a production environment.

    Of course, integrating application testing within a DevOps process enables organizations to identify more potential issues earlier when they are easier and less expensive to fix. The challenge has always been getting developers to appreciate the value of testing early and often in the application development process.

    Regardless of how application testing continues to evolve in the age of DevOps the one thing that is certain is no matter how automated it gets developers are going to be more involved than ever.