Category: Features

  • Meet the New(ish) Pipeline

    Meet the New(ish) Pipeline

    My company keeps a sample project running pretty much all of the time. Our current project (multi-target web/Android app using Java and Spring built with Gradle and Ant that we’ll call “Finder” here) is getting a bit long in the tooth, so we have been casting about for a new one over the last several months. From non-work sources, we came upon the idea of multi-targeted Godot with C# as a viable side project. Moving to a completely different set of tools is kind of the point; it makes us stretch, and that is the idea of our ongoing projects. The stated goals for these projects are:

    1. Stretch knowledge about different languages, DevOps tools and platforms.
    2. Keep doing what you all are doing. Too many pundits have “grown out of” hands-on work, and it shows.
    3. Try out new ideas/architectures.
    4. Avoid “kitchen sink.” We can’t have a current test app that includes all available technologies, so keep it streamlined. Approach this as a production app that we use for testing, not as a test app.

    That’s why we’re looking to replace our Java/Spring project. We’ve got that pipeline down, we’ve tweaked the app until we are happy with the core and it hasn’t changed in a good while. But change is a part of DevOps, so we’re off.

    The Obvious

    Let’s take care of the obvious elements in our toolchain right away:

    1. Git. We’ll use GitHub, but it was a given that some form of Git would be holding our code.
    2. Visual Studio Code. We’re doing C#, so this is going to be the most useful option for code editing. The built-in Godot editor is good and will be used, but VS-Code offers C# functionality add-ons.
    3. Antsle is the deployment tool of choice, though this has changed over the years. We’ll use it to set up install sets for clients.
    4. Jenkins or Hudson. We’re going to use this only for test management.

    The Less Obvious

    There are several constraints we want to (or have to) meet. First, we want to avoid using Jenkins or Ant to control the entire build process, mostly because we’ve done both in previous projects, and most DevOps teams are no longer using these methods for new projects. Second, this is a sample we keep, not an investment, so we want to use open source technology if at all possible, to keep expenses down. Third, we’re using a game engine to do app development. This has some impacts–like most game engines, it is in control at runtime and we need to bundle the engine in the final application.

    1. The Godot Engine. Godot is written as a game platform, but its graphics/cross-platform abilities make it useful for graphics-heavy apps like ours. But we have to include the engine and its config files in the pipeline.
    2. Code Scanning. Static analysis was a late addition to Finder, and it’s in Java. This project is in C# and Godot Script. We have not yet found a tool that supports Godot Script, but C# we can get; it just adds restrictions. So we will use a static analysis tool, but we’re still deciding which.
    3. Build Automation. We’ve used Jenkins, Gradle, Maven and Ant to handle this step at different times in the past, but something newer would be a good idea. We are still evaluating this step. We will likely end up back on one of those tools because they are still used today. Godot is a new element to us, so we are also evaluating what is required to hook into it and make functioning apps in a repeatable process.

    Fueled Innovation

    We’re having fun—first couple sprints have us running a working prototype, so we’re happy with that. There’s still a lot to implement in the app and in the pipeline, so we’ll no doubt have more to say here soon. We’re talking about putting a server component in—not because the original concept needs it, but because our outgoing app—Finder—had server and client sides, and our need for things like Postman go away without this architecture. And yet, API security, testing, etc. is an important part of the DevOps architecture. As you can see, we’re still iterating.

    As to Finder, we normally take down the pipeline for a previous playground project when the new one goes up, but Finder costs us little. There are five containers (three permanent for the app hosting; two for build/test) and some processing power, so we’ll probably keep it since it is using a different set of pipeline tools (and languages—it is mostly Java/JavaScript).

    Meanwhile, you all keep rocking it. We’re building an(other) app that no one but us will ever see—but you are keeping the enterprise humming. Kudos to you!

  • New Relic Survey Shows Greater Appreciation for Observability

    New Relic Survey Shows Greater Appreciation for Observability

    A global survey of 1,300 software engineers, developers, IT leaders and executives found half of respondents (50%) are in the process of implementing an observability platform. Another 23% of respondents said that they cannot gain end-to-end observability at all, while 26% said they believe they have mature observability practices in place. The survey, published today, was conducted by CITE Research on behalf of New Relic.

    The majority (94%) of IT professionals said observability is important for their job role, while nearly three-quarters (74%) said there is still room to grow their observability practice. In fact, 81% of the 450 C-suite executives that participated in the survey said they expect to increase observability budgets in the next year, with 20% expecting those budgets to increase significantly.

    Overall, the survey finds that lack of resources (38%), skills (29%), understanding of the benefits (27%) and strategy (26%) are top barriers to achieving observability.

    Observability, in one form or another, has always been a core tenet of DevOps best practices. Most DevOps teams have relied on monitoring tools to observe a specific process or function. The challenge is those monitoring tools track a pre-defined set of metrics such as resource utilization. Ideally, observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshot issues without having to rely solely on a limited set of pre-defined metrics.

    Nearly three-quarters of respondents (72%) noted that IT teams today need to toggle between at least two different tools to manage IT, with 13% having to use more than ten different tools to monitor the health of their systems.

    In contrast, observability platforms make it possible to correlate events across both applications and infrastructure. This capability makes it easier for analytics tools to identify anomalous behavior indicative of the root cause of an IT issue. Armed with these insights, it becomes a lot simpler for IT teams to resolve issues faster.

    The New Relic survey notes that 60% of respondents still monitor telemetry data at the application level, which indicates that a massive amount of valuable telemetry data is largely going unused. A full 91% of IT professionals that responded to the survey noted that observability is critical at every stage of the software life cycle, especially when it comes to planning and IT operations.

    Top benefits cited include faster deployments (27%), more cost-effective use of IT resources (25%), better digital experiences (23%) and improved productivity (22%). The top use cases cited involved digital transformation initiatives (42%) followed by cloud optimization (37%), DevOps processes (36%) and bringing products to market faster (36%).

    Buddy Brewer, group vice president and general manager for New Relic, said observability will become a more pressing issue in IT environments, thanks to the rise of microservices-based applications running on Kubernetes. A full 88% of IT decision-makers New Relic surveyed said they are exploring Kubernetes and containers at some level right now, but only 10% said they have deployed them in a production environment. A total of 40% expect to be in production within three years.

    In general, it’s become critical for IT teams to be able to query data in real-time to resolve issues that are becoming more challenging to resolve at a time when organizations have never been more dependent on IT. Sampling data in the hopes of discovering issues is no longer sufficient, noted Brewer.

    The challenge, of course, is that most IT teams have been using legacy monitoring tools for so long that it’s still hard to imagine there might be a better approach. That doesn’t necessarily mean all those legacy tools will be immediately replaced, but it does mean many of them may simply fade away over time.

    One way or another, most IT teams will soon be able to observe IT environments more deeply. Developers increasingly are taking advantage of open source tools like OpenTelemetry to instrument applications in a way organizations can more easily afford, regardless of whether they fully appreciate the nuances of modern observability versus traditional monitoring.

  • MediaOps, Boca Code Award ‘Engineer the Change’ Scholarship

    MediaOps, Boca Code Award ‘Engineer the Change’ Scholarship

    I am really proud to announce that, in partnership with Boca Code, MediaOps has awarded a scholarship for the Fall 2021 term to the Software Engineering Career Course, a 10-week rigorous and fast-paced coding boot camp from Boca Code. Alina Maliuk, a first-generation immigrant from Ukraine, is the recipient.

    The Engineer the Change scholarship was initiated by MediaOps to help foster diversity in the local tech industry and empower individuals from underserved communities to develop the skills needed to succeed in IT. Boca Code offers comprehensive training in software development to enable course attendees to develop these skills. 

    Need for Diversity in STEM fields

    Research shows a staggering lack of diversity in STEM fields, most noticeably within computer and engineering positions. The tech industry is predominantly white and male, which appalls me (yes, even though I am a white male), and I wanted to do something to encourage more diversity across our industry. Gender and race inequity are glaring workforce disparities that require both recognition and responsibility from those at the highest levels in the tech world to create a more diversified and equitable workforce.

    Two Full Scholarships Awarded Annually

    MediaOps sponsors two $10,000 scholarships each year to enable students to enroll in Boca Code’s flagship Software Engineering Career Course. The full-time, 10-week intensive course combines theory and hands-on, project-based learning to prepare students for a career in software engineering. As part of their coursework, students take on development projects for local businesses, providing the students immediate, hands-on experience with real projects.

    The Engineer the Change scholarships combine two goals that are core to the MediaOps team and me. The first is doing our part to make South Florida a world-class hub for the tech industry. The second is providing opportunities for individuals from disadvantaged communities to acquire the skills they need for a successful career in tech. My hope is that we also serve as an inspiration to other local companies to fund similar programs in support of these worthy goals.

    About Our Winner, Alina Maliuk

    Alina Maliuk scholarship
    Alina Maliuk, Engineer the Change scholarship winner

    Alina hails from Ukraine, where she earned a bachelor’s degree in management. After college, she married and had her first son. During her maternity leave in 2012, her family emigrated to New York. Being first-generation immigrants, they had to start their life here in the United States from scratch. For the first few years, Alina and her husband worked in entry-level jobs. In 2018, now a mother of two, Alina took several online classes in web development and rediscovered her love of learning—particularly software development. Now a resident of Boca Raton, Alina found Boca Code and attended a workshop in web development.

    Alina was energized by that workshop, which furthered the continuing education she had already begun. Through that initial course, the Boca Code team inspired her to go even further. She is ready to move on to her next career, which she wants to be in software engineering. Thanks to the Engineer the Change scholarship, she will now have an opportunity to accelerate her tech career. At the same time, she is demonstrating to other female immigrants in Florida that if you can dream it and work hard to achieve it, you can make it happen.

    Alan Shimel

    CEO, MediaOps

    Additional Resources

  • JFrog Acquires Upswift to Push DevOps to the Edge

    JFrog Acquires Upswift to Push DevOps to the Edge

    JFrog today revealed it has acquired Upswift as part of an effort to extend its DevOps reach to edge computing platforms. Terms of the acquisition were not disclosed.

    Upswift has created agent software that is deployed as a binary file on an edge computing platform that makes it possible to continuously deliver software across a fleet of edge computing devices and platforms. That approach makes it possible for organizations to deploy software regardless of whether the target platform is only running Linux, containers or a virtual machine.

    JFrog CEO Shlomi Ben Haim said the agent software developed by Upswift will be integrated with the continuous integration/continuous delivery (CI/CD) platform created by JFrog to make it easier for organizations to manage the full life cycle of application workloads deployed on edge computing platforms.

    JFrog Upswift

    Organizations that deploy software on edge computing platforms and devices today typically have massive backlogs of updates that they are unable to deploy in a timely manner, said Ben Haim. By connecting the Upswift agent software to a (CI/CD) platform designed from the ground up to support binaries, it becomes possible to reduce that backlog as part of what JFrog describes as its ‘liquid vision’ for deploying software, he noted.

    In addition, after acquiring Vdoo to gain access to a set of analytics tools that discover vulnerabilities in application binaries, JFrog will also be able to bring DevSecOps best practices to edge computing applications as well, noted Ben Haim.

    The number of application workloads being deployed on edge computing platforms is rising sharply as organizations seek to process and analyze data closer to the point where it is being created and consumed. In fact, that shift is at the core of digital business transformation initiatives that depend on edge computing platforms and devices running, for example, internet of things (IoT) applications to process data in near-real-time.

    Upswift CEO Amit Ezer said it has already become apparent that the existing processes of updating software on edge computing platforms and devices are capable of handling that influx of software. Upswift created its tool to enable developers to assume more control over those processes, as responsibility for managing and securing software continues to shift left, he added. The issue in many cases is that organizations lack the ability to consistently deploy software on edge computing platforms, which results in a hodgepodge of homegrown tools being created to deploy software on edge computing platforms using largely manual processes.

    It’s too early to say just how much software will be deployed on the edge, but there may come a day when there is more software at the edge than there is in the cloud. Of course, the real challenge will be unifying the management of DevOps processes, regardless of where any given piece of software might be deployed. In fact, as edge computing continues to evolve, the line between edge computing and cloud computing will continue to blur until they are little more than natural extensions of each other.

  • Google Donates Sqlcommenter to OpenTelemetry Project

    Google Donates Sqlcommenter to OpenTelemetry Project

    Google announced today that Sqlcommenter, an open source project that provides access to an object-relational mapping (ORM) auto-instrumentation library, is being merged with OpenTelemetry, an open source project for building agent software to instrument applications.

    Nimesh Bhagat, a product manager for Google, said Sqlcommenter is designed to make it simpler to correlate application and database telemetry. OpenTelemetry provides a mechanism for collecting logs, metrics and traces that is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The challenge organizations using OpenTelemetry to collect that data will face is the need for tools to query that data in whatever platform it ultimately resides, noted Bhagat. OpenTelemetry previously lacked a common standard by which application tags and traces can be sent to databases and correlated with an application stack. Google envisions that IT teams will either employ Sqlcommenter to explore data or access via platforms such as Cloud SQL Insights or a third-party application performance management (APM) platform.

    Google made Sqlcommenter available as open source software earlier this year. It is designed to enable object–relational mappings (ORMs) to augment SQL statements before execution by, among other things, providing access to comments containing information about the application code that caused its execution.

    Sqlcommenter also allows OpenTelemetry trace context information to be propagated to the database, enabling correlation between application traces and database query plans that ultimately should make it easier to, for example, determine the root cause of a performance issue involving a microservices-based application making calls to a database. Sqlcommenter is primarily designed to be employed by full-stack developers that have some familiarity with how to use SQL to query data residing in an observability platform, noted Bhagat.

    Specifically, Sqlcommenter is designed to provide deeper insights into the performance of backend databases by simplifying the process of correlating slow queries with source code. It currently supports Python, Java, Node.js and Ruby languages along with ORMs such as Django, Sqlalchemy, Hibernate, Knex, Sequelize and Rails.

    As the number of applications that are instrumented using open source software continues to steadily increase, the amount of data that a DevOps team could potentially query also will steadily increase. Cloud storage platforms have made it easier to store that data at an affordable cost but the next challenge is clearly going to be finding a way to surface meaningful insights from all that data. Undoubtedly, achieving that goal will require everything from query tools to machine learning algorithms capable of identifying patterns at a level of scale that would not otherwise be possible.

    In the meantime, DevOps teams can look forward to being able to attain a level of observability that previously could only be achieved at a cost that tended to limit the number of applications organizations would instrument. At a time when the dependencies between applications increases with each passing day, that level of observability will soon prove indispensable.

  • How Does That Stack Up?

    How Does That Stack Up?

    There’s an entire class of successful innovations made up of things that people didn’t realize they needed. Things that were an obstacle they found a way around and didn’t think about until the innovation showed them an easier way. Think of things like virtualization (hardware was the bottleneck, but “That’s just the way it is,” until it wasn’t) or web monster shopping sites (“They don’t have that in the stores; I guess I’ll live without,” until you could order the weirdest or smallest things from the likes of eBay and Amazon).

    My rule here is that I only talk about products and services if I use them and find value in them. And even then, I try to do it sparingly. But in the spirit of the innovation story you just read, I’m going to point you at one of the coolest new websites on the net. I don’t know the company; have not done business with them, couldn’t even name a principal. But I have spent a surprising amount of time on their site, both in the Feed section and Browsing Stacks.

    The site is Stackshare.io, and the reason I think it is so innovative is what and who it is aimed at: Listing companies’ DevOps stack and offering advice on different parts of the DevOps stack.

    The Browse Stacks section is for presenting what other organizations are doing, so you can get good ideas for your own stack. But the developers of the site appear to have recognized the other useful aspect of this information: Matching organizations to prospective employees based on technologies. You can go look and see that Company X is using GitLab, for example, and that can then influence your job search.

    The Feed section is just a running discussion that uses categories to ask questions or offer insight and advice about decisions a given organization came to—so, “Chose X over Y” is actually a standard subheading for posts, as is “Need advice on Z” or “Recommends A”. It makes for great reading, and no doubt while reading through, you’ll have some advice to offer one of the “Looking for advice” posts.

    The Stackshare.io participants seem to range in size from startup to massive, and that is one of the things I hope they change as they grow—the options for looking at DevOps stacks are “Popular” and “Trending”. We all want more filters than that—organization size, number of apps, stuff like that—to help us zero in on environments that are similar to our own. My little company does not have the needs that Facebook does. And while it’s true that some startups use more tools in their toolchain than Facebook, you get the idea.

    Another issue I had with Stackshare.io’s site (that I fully understand, but still wish could be different) is forced registration/login to see some data. It’s not even particularly sensitive data—I know what ELK and Azure Data Services are, so putting the definition/description behind a registration page/login is kind of silly. But every free site needs data to pay the bills, and I suppose I’m glad it is info that’s also available elsewhere that is behind a reg wall.

    The company offers APIs for integration to your site/tool and private company instances of StackShare to help you manage one of my other gripes—the five (or fifty!) different DevOps stacks many organizations have allowed to develop in-house.

    So if you are job hunting, Stackshare.io can help you find companies doing what you would like to do or know you’re good at. If you are an organization trying to improve or set up a new toolchain (which is how I found the site), it has a lot of information to digest, though not all of it is easily accessible at this point. Either way, keep kicking it and keep an eye out for tools that do the job, no matter where you are looking.

  • Progress Expands Chef Automation Portfolio

    Progress Expands Chef Automation Portfolio

    In advance of its ChefCon 2021 conference, Progress today unveiled a series of enhancements to its Progress Chef automation platform that includes managed services and a software-as-a-service (SaaS) edition of its offerings as well as tighter integration with cloud services and other third-party tools.

    Sudhir Reddy, vice president of engineering for the Chef business at Progress, said the SaaS platform, now available in beta, combined with the managed services provided by Progress will make the automation platform much more accessible to a wider range of organizations.

    Progress is now also extending the reach of its Chef Compliance Automation for Cloud Resources offering across Amazon Web Services (AWS), Microsoft Azure and Google Cloud Platform (GCP) to include Chef premium content that aligns with best practices for each cloud environment in addition to providing integration with Azure Policy, a suite of tools from Microsoft that compares resources to a set of business rules that define how cloud infrastructure should be employed.

    Chef

    Finally, Progress has enhanced Chef Enterprise Automation Stack to provide tighter integration between compliance and infrastructure automation along with adding infrastructure operations dashboard, support for Windows patch management, integration with third-party secrets management tools, and improvements to Chef Workstation and Chef Test Kitchen to simplify testing within application development processes.

    With the release of a SaaS platform and the availability of managed services, Chef is joining a long list of IT vendors that are removing responsibility for deploying and securing their platforms from internal IT operations teams. It’s clear more organizations now want to spend more time on building and deploying software than managing the platforms that automate those tasks on their behalf, said Reddy.

    It’s not clear how many IT teams that have already deployed a platform might shift over to a SaaS platform or rely on an IT vendor to manage that platform on their behalf. However, in the wake of the COVID-19 pandemic, many IT teams continue to work from home to help combat the spread of the virus. Many of those organizations have either shifted platforms to the cloud themselves, adopted SaaS platforms or outsourced the management of IT platforms altogether. There are also many small organizations that simply don’t have the internal expertise and resources required to stand up an IT automation platform on their own.

    That approach also makes it simpler to converge security, compliance and IT operations management within the context of a larger DevSecOps workflow, noted Reddy.

    The great paradox of IT automation has always been the level of expertise required to achieve it. Organizations that lack the expertise required to create automated playbooks, for example, don’t derive much value from an IT automation platform. It’s only been larger companies that have mostly derived the most benefit from investments in IT automation. However, as IT automation becomes more accessible via SaaS platforms and managed services, the level of expertise required to benefit from it naturally drops. In fact, with the advent of artificial intelligence (AI) for IT operations (AIOps), many manual IT tasks are increasingly being automated.

    It’s too early to say whether the IT industry as a whole is on the cusp of the next wave of automation, but the raw ingredients that make up the primordial soup needed to achieve that goal is, at this point, only awaiting a catalyst.

  • Enterprise Blockchain Adoption Hinges on DevOps

    Enterprise Blockchain Adoption Hinges on DevOps

    There is a lot of excitement around blockchain right now, for a good reason. Production use cases continue to emerge across sectors such as manufacturing, energy, air travel, insurance and finance. Gartner predicts that by 2023, organizations using blockchain smart contracts will see a 50% increase in their data quality.

    Yet, Gartner also predicts that blockchain will reduce data availability by 30%. Before its use can reach ubiquity, blockchain-based application development has some hurdles to overcome. And if enterprises are to get on board, executives will need to see tangible proof of its value in the form of reduced expenses and increased profits directly correlated with blockchain adoption.

    “Enterprises aren’t going to change their business model to adopt a new technology,” said CasperLabs CEO Mrinal Manohar. Just as it’s difficult to dramatically alter the UX for established products, it’s similarly difficult to change the developer experience for established software production routines.

    I recently met with Manohar to discuss the limitations and the changes required to accelerate enterprise blockchain adoption. According to Manohar, much of it boils down to taking a DevOps approach—reducing energy use, enabling CI/CD, ensuring stable development environments and providing room for change and ongoing testing—will be critical to spur adoption. This could also mean abstracting complexity away from the developer and end users. However, once more production proofs are out there, the shift toward blockchain will be swift and it will be massive.

    What’s Hindering Enterprise Adoption?

    Enterprise adoption comes down to UI and UX, said Manohar. He predicts blockchain will follow a similar path as the internet. In its command-line days, the internet was only adopted by die-hard techies. But once visual-based actions were possible through the browser, consumer interest exploded.

    Similarly, blockchain might see more production use once the complexity is pushed to the background. “A technology is awesome when it’s 99% invisible,” said Manohar. However, at the moment, blockchain is 99% visible, he adds. For example, it isn’t easy to spin up a node and test a blockchain app. Lowering the requirements to 50% with more abstraction, simulation and integration with cloud providers could help tremendously.

    In general, Manohar believes that four key factors are hindering enterprise adoption:

    1. Enterprises need a lower energy footprint. Attaining high throughput on a public network without sacrificing decentralization and security is tricky. If blockchain drains operations when scaled, organizations may simply opt for a centralized data store. Therefore, enterprises will require a blockchain with minimal energy requirements.
    2. Large companies want developer environments that are similar to what’s already in place. In many ways, blockchain throws out the old playbook. But such a significant shift is a hard sell across multiple IT divisions. Large companies will require blockchain abilities that plug into the existing developer stack. Using a blockchain with familiar programming languages and plugins to popular IDEs like Visual Studio and IntelliJ will decrease onboarding effort.
    3. They need more control. Blockchains typically use immutable contracts. This is great for sustaining a shared distributed history, but Manohar believes that contracts need to be more malleable to enable continual upgrades. Furthermore, there is the issue of privacy when adopting blockchain. Some implementers don’t want all 100% of data to be on the public chain. Having the ability to maintain both private and public nodes for smart contracts could help enable more flexibility.
    4. Enterprises want ongoing support. Some companies will be autonomous using open source, but most enterprises will not adopt new or emerging technology without a support email attached to it, said Manohar. Thus, there may be a need for professional services and outsourced blockchain experts.

    Applying a DevOps Perspective

    Another big boon to accelerating blockchain adoption could be considering a DevOps perspective. To Manohar, introducing DevOps into blockchain requires more continuous updates, mitigations to avoid breaking change and simulations to enable better testing environments.

    An immutable blockchain infrastructure seems to be at odds with the DevOps creed of rapid deployment. Thus, Manohar said that choosing a blockchain that enables more mutability will help upgrade contracts and allow functionality to change over time. For core infrastructure changes that affect the entire public chain, the majority of validators must, of course, accept upgrades before they are distributed.

    Simple functionality, like a push notification, could ease this process to allow a quick Yes or No response from operators.

    Another potential concern is backward compatibility. What if the aggregate introduces significant breaking change? Manohar said that while this is technically possible, it’s improbable. “That would be a controversial upgrade,” he said, and validators have an incentive to keep a sturdy system intact. “They only do it if they think it’s better for the long-term health of the network.” But guardrails on this process will help avoid edge cases. Thankfully, if a change did disturb a minority of users, a blockchain can be forked. We’ve already seen this occur with Bitcoin Cash and Bitcoin, or Ethereum Classic and Ethereum, for example.

    Another shift that will help enterprises embrace DevOps in their use of blockchain will be better testing and deployment processes. It’s notoriously challenging to test blockchain applications while they are tapped into the broader network. Thus, engineers could benefit from a simulated runtime wholly divorced from the main network for testing purposes.

    The Future of Blockchain Development

    So, what does the future of blockchain enterprise adoption look like? Well, it appears we will continue to see new novel use cases emerge that use blockchain in innovative ways. For example, Manohar pointed to the use of blockchain to power an NFT digital twin, a patent marketplace, a tagging system for supply chains and the mortgage processes for a core banking system. Within these scenarios, he noted how blockchain is exorcising cost in many ways; smart contracts are being used to avoid legal fees, eliminate manual processing and improve fault attribution.

    Before blockchain can go mainstream, however, work must be done to bring other enterprises into the fold. Namely, it will necessitate more abstraction of the infrastructure layer. In this effort, low-code integration capabilities could help make API connections to ERP systems or cloud service providers more accessible.

    To inspire adoption, the economy must also change its perspective on what constitutes blockchain success. As of now, common KPIs are surface-level, like: How many wallets? What is the trading volume? Instead, Manohar believes we need to focus on proving actual cost savings from blockchain adoption. Once accountants are publicly reporting revenue gains, shareholders will demand that every company do this, he said.

    “Once that catalyst happens, adoption will happen very, very fast, and demand for blockchain in enterprises will be massive.” At that point, some years down the line, blockchain may truly power a new economy.

  • CircleCI Bets on Webhooks to Streamline DevOps Integrations

    CircleCI Bets on Webhooks to Streamline DevOps Integrations

    CircleCI today announced CircleCI webhooks, a feature that enables software engineering teams to build integrations that react to CircleCI job and workflow status notifications in near-real-time.

    Tom Trahan, vice president of business development at CircleCI, said this latest iteration of the company’s implementation of webhooks, which send messages any time an event occurs, will become the foundation upon which CircleCI will expand the ecosystem surrounding its continuous integration/continuous delivery (CI/CD) platform.

    For example, the CircleCI platform can now employ webhooks to integrate with a CI visibility tool from Datadog that can now be used to surface CicleCI metrics. Sumo Logic is similarly employing webhooks to enable its Continuous Intelligence observability platform to track events on the CircleCI platform. Those capabilities are now available via the software-as-a-service (SaaS) edition of the CircleCI platform.

    Trahan said the primary goal is to reduce the amount of friction within DevOps workflows that span multiple platforms by increasing the level of orchestration DevOps teams can take advantage of without having to create and maintain integrations using low-level application programming interfaces (APIs).

    Webhooks make it possible to streamline developer workflows while also increasing overall engineering velocity, he added. In addition, it should also become simpler for DevOps teams to triangulate events in a way that results in incidents being resolved faster, noted Trahan.

    CircleCI is relying on webhooks to strengthen its case for a best-of-breed approach to DevOps that enables organizations to mix and match tools as they best see fit rather than getting locked into a single integrated DevOps platform. The best-of-breed approach often makes it simpler to take advantage of new tools as they become available. The downside is the level of integration that DevOps teams need to maintain after they integrate all the tools that make up a DevOps workflow.

    It’s not clear what impact relying on webhooks to share messages in real-time about events as they occur across multiple platforms will have on integrations. However, any effort to streamline DevOps environments without requiring DevOps teams to abandon one tool for another is likely to be well-received. After all, every minute spent managing a DevOps platform means much less time that can be applied to building and deploying applications.

    It will be up to each DevOps team to decide to what degree a best-of-breed approach to DevOps suits their organization. As more organizations realize how dependent they have become on software to, for example, drive digital business transformation initiatives, many IT leaders are making a case to standardize DevOps platforms across their organization. Achieving that goal, however, may be difficult given all the various departments within an enterprise that have already adopted a wide range of tools and platforms.

    In the meantime, the focus on developer and engineering productivity will only increase as organizations look for ways to further accelerate application development and deployment at a time when software updates can now have a profound impact on profits and revenue. In fact, a delay to one software project can have a cascading impact on a wide range of initiatives that can’t go forward because of their dependency on a delayed project. Given the impact all those delays might have on revenue, the tolerance organizations have for delays to software development projects is understandably lessening by the day.

  • That Other Continuous: Data Protection

    That Other Continuous: Data Protection

    Looking around at things to add to our current project’s toolchain (that is one of the reasons we do these side projects; to check out all sorts of new ideas/tools/etc), and researching the company where a former coworker is the CMO, I was drawn into the resurgent world of continuous data protection (CDP). It is interesting that we could end up adding CDP to the list of ‘continuous’ activities, like CI/CD/… CDP, even though CDP was named long before any of our DevOps or Agile terms were born.

    The whole idea of CDP is that it is always backed up, so you are continuously protected. It largely failed when first introduced because it massively increased both the cost and complexity of your storage environment. Essentially, it required a hot copy of all of your storage. Not something most enterprises were willing to invest in. I mean, it was friggin’ expensive. And it used virtualized storage, which no one wanted to mess with. Long story short, at the time, virtualized storage introduced issues no one wanted to deal with.

    But several things have changed over the years, and it is interesting to see where CDP concepts have cropped up. You see, today it is being used to protect companies from ransomware. It’s still a bit of complexity, but we’ve become experts at hiding complexity. It still requires a lot of storage (I would argue that the current versions I’m seeing require more storage than the original CDP solutions did), but when your environment is taken over by ransomware, a clean machine can be used to restore data to a point before the ransomware took over. From a “continuous protection” standpoint—giving it a modern meaning along the lines of “continuous testing”—that is an astounding feat.

    The way it works—at least the way that the current implementations I’ve looked at work—is pretty simple at heart, though as my former coworker likes to say, “Easy in concept, difficult in implementation.” It keeps every write ever made to storage, with time and currency markers. So it knows what the disk currently looks like, but it can also get to what the disk looked like at 12:33 am yesterday. Or January first, or a year ago… You get the idea.

    It uses a ton of space because there is an implied truth here—nothing is destroyed. In traditional storage, if you change a document and hit save, the document is written over. In the new iteration of CDP, if you change a document and hit save, the changes are recorded, but the original is not altered. Call it “non-destructive writes,” if you will.

    It is worth checking out. The concepts are huge, and offer protection that is otherwise in short supply. If you have a dedicated DevSecOps team, they’re the ones I’d point at it, since the big reason for using this is a security reason at heart (though there are other uses; I’m currently gushing about security because this is a big deal).

    I’ll get back on track and report back about things directly in the toolchain for our newest DevOps project next week. I thought this was worth writing about, since most of you all could use the concept, even if only for peace of mind. You’re rocking it! Protecting your work against the newest range of attackers is just common sense. It’s worth seeing if the modern iteration of CDP is for you.