devopsy.kinsta.cloud

Author: Bill Doerrfeld

  • Report: The State of DevOps Automation

    Report: The State of DevOps Automation

    In the race to accelerate digital transformation initiatives, organizations are encountering more incidents, more downtime, and longer resolution times. In fact, 90.4% of organizations saw an increase in incidents since the pandemic began, according to a recent Transposit report.

    In a completely digital economy, service downtime takes a higher toll. For ITOps teams, working remotely also requires new strategies to organize a team’s response to issues. Amid rising incidents, 94.1% of organizations have increased their focus on service reliability engineering (SRE) practices.

    Transposit just released their 2021 State of DevOps Automation report, which surveyed over 500 IT, DevOps, and SRE professionals across the U.S. The report sheds light on the state of remote work as it relates to incident response processes, reveals increased demand for SREs and highlights the automation they’re using to narrow the resolution time gap. Below, we’ll explore the report’s key takeaways and discover what sort of DevOps automation is trending.

    Remote Work Slows Incident Response

    Try as they might, videoconferencing and group chat solutions like Zoom and Slack just can’t compete with in-person collaboration. According to the survey, 93.6% of ITOps and software teams reported it takes longer to resolve incidents while working remotely.

    Since the pandemic began, 97% of teams have adopted remote work, and 90.4% have experienced increased service incidents. It doesn’t bode well that incidents are increasing as incident response time lags.

    The report findings frame remote work as a top hurdle to resolving issues — 46.1% reported that not being physically in the same room to collaborate on fixing issues was a top concern. But remote work is only one element; others cite changing processes and a lack of visibility into who’s responsible for maintaining code as another challenge.

    Incident Response Hangups

    To make up for these concerns, organizations are investing in incident response strategies. The report found that 56.4% of companies are adopting communication and collaboration tools, with automation tools at 52.3% and integration tools at 43.9%.

    Though the report showed greater investment in those tools, it also identified a persistent communication gap. When conducting postmortems, it can be challenging to correlate human action to incident responses. More than half (57.4%) of respondents felt it’s hard to piece together human actions and team communication during incidents. This context is essential to make sense of mean-time-to-response (MTTR) metrics.

    Interestingly, the report found 96.4% of respondents believe mining insights from human data, like Slack channels or group emails, could help improve incident responses. It appears that having a clearer way to harvest and query incident-related data throughout the entire response process is a universal concern.

    SRE Goes Mainstream

    As I previously reported, the SRE role continues to evolve and gain favor. The report similarly hypothesizes that, based on the survey results, the SRE title is going mainstream. Out of 295 respondents surveyed, 98% said they increased SRE practices in the last 12 months, and 62.4% said they would be expanding SRE efforts in 2021.

    In 2021, SREs will likely remain a hot commodity in hiring. However, these individuals will face mounting complexity in addressing real-time incidents for complex distributed systems. They will also likely be tasked with automating manual processes, as 51.7% of respondents reported that a lack of automation is the top cause of slow incident resolution.

    Automation Goals and Barriers

    40% of companies surveyed now have at least one dedicated full-time staff member working to create in-house tools or bots to automate incident responses. So, what sort of automation is common in the SRE field? The report shows SREs constructing custom scripts to automate things like:

    • Customer communication
    • Recording MTTR metrics
    • Automating infrastructure
    • Corporate communication
    • Auto-generating runbooks
    • Creating or updating tickets

    But, automating these processes is not all that easy. As a top barrier to automation, DevOps professionals collectively cite a lack of documentation. It seems by replacing tribal knowledge with up-to-date, shared documentation, teams could quickly unlock some gains. So, make documentation accessible, and provide copy-and-paste abilities for sample scripts.

    SRE and Automation to the Rescue

    The study demonstrates how the remote economy has increased both the number of incidents and the mean response times to address them. To fill this gap, organizations increasingly turn to site reliability engineering practices to automate incident responses. By leveling up DevOps with SRE, teams could reduce errors and improve overall availability as development teams progressively deliver new features.

  • Low-Code Makes API Testing More Accessible

    Low-Code Makes API Testing More Accessible

    APIs are rising both in quantity and business value. But with that growth comes an increasing number of vulnerabilities, which could affect cloud service dependencies and internal APIs. Functionality and uptime could also suffer as cloud services evolve their APIs with new versions. Thus, API endpoint testing is critical to ensure a reliable back end. With growing interest across the organization to stabilize these integrations, how do we test APIs in a reproducible way that non-developers can grasp?

    Low-code platforms — tools that enable workflows with visual programming models — are opening doors for citizen developers to construct applications in a visual mode. Similarly, the low-code concept could aid API testing by providing an easier method to create API tests to verify functionality and uptime. This could arm quality assurance, DevOps and site reliability professionals with an easier means to test high-value integrations and their UI representations.

    Amid a growing headless movement, APIs are now powering many end-user experiences. However, during a developer drought, it may be challenging to find talent with integration know-how.

    I recently met with Dan Belcher from Mabl to discuss combining low-code and API testing. Here, we’ll uncover why it’s difficult for non-engineers to conduct API testing and consider what API testing must address to ensure today’s software meets tomorrow’s digital innovation demands.

    A Low-Code Approach to API Testing

    The software landscape has evolved rapidly in recent years. “Many forces have combined to create an incredible amount of change,” said Belcher, citing DevOps, CI/CD, the API economy and agile enterprises as examples. In this era, applications are becoming more dynamic, disrupting the way teams conduct end-to-end quality testing.

    API-based cloud services are part of this new frontier, becoming popular building blocks within modern applications. Yet, integrations require upkeep as cloud services evolve and version. Without continuous testing of API-based cloud services, organizations could easily suffer from breaking change and poor usability.

    Unfortunately, as it stands, typical QA teams aren’t well-equipped to handle ongoing regression testing for APIs. Instead, “developers, historically, have been the only ones that understood API testing,” explained Belcher. For years, API testing has been relegated to the realm of developers, who use tools like Postman to perform unit-level testing. But why is API testing so difficult for others?

    What Makes API Testing Difficult?

    Well, to start, non-engineers are typically more accustomed to UI-based testing. These environments use a structure designed for anyone to consume, making it easy to locate fields and confirm values, Belcher explained. When it comes to API testing, you often must handle massive chunks of JSON responses to match fields and surface issues. Testing API responses typically requires writing code, parsing JSON and writing it to variables, Belcher explained, which may be difficult for non-developers to implement.

    Or, difficulties could arise when testing asynchronous endpoints, Belcher added. Asynchronous APIs often involve background server processes that take a while to respond. This comes at a moment when event-driven architecture (EDA) is a growing trend. Take AsyncAPI, a standard API specification for EDA APIs, which recently joined the Linux Foundation. A recent report found AsyncAPI production usage tripled from 5% in 2019 to 19% in 2020. According to Belcher, testing such asynchronous services requires an infrastructure for continuous testing to anticipate these emerging scenarios.

    Four Crucial Areas For API Testing

    So what should API tests be focusing on? Belcher highlighted four areas that warrant close examination throughout the API testing process:

    • Functionality: This is the core functionality. Does the API do what it promises? Does it return the correct data?
    • Uptime: What is the API’s availability? Are all endpoints available in production?
    • Speed: This ensures cloud dependencies meet SLAs. What is the latency? Is there drift in API performance over time? Though API monitors like Api.expert help aggregate industry performances, they might not always reflect your specific customer-level realities. Belcher recommends not placing faith in supposed performance and instead, clocking your own.
    • End-to-End: Belcher stresses the role APIs now play across customer-facing presences. An end-to-end workflow should involve API endpoints, how they interact with one another, and the resulting visual representations. “Just like we have end-to-end testing for UIs, we should have end-to-end testing for APIs,” he said.

    API testing isn’t always combined with other parts of the user experience, but Belcher believes, going forward, end-to-end testing will be critical to ensure all workflows function in concert. For example, if a user profile is created within a browser, teams should verify the user profile has been created at an API level, as well.

    Low-Code API Testing and Shift Left

    APIs are beneficial to avoid reinventing the wheel. However, 100% operational uptime is impossible. Applications will inevitably encounter API problems, including outages, technical failures and functional misrepresentation. Developers also often come to rely upon production behaviors that may not be officially documented, an effect known as Hyrum’s Law. This can cause unstable integrations as APIs version over time.

    Thus, for these reasons and more, it’s a good practice to adopt continuous API testing. With the surging interest in site reliability engineering, spotting API errors early on will become an important aspect in reducing mean time to recovery (MTTR). Testing third-party APIs could also expose production latencies to inform cloud investments.

    “As the pace of change is going up, how do we deliver great quality to our customers?” Belcher asked. API testing is one answer to address this rapid change, and is growing in importance for the software delivery life cycle. It especially rings true for headless frontend experiences, where the API-driven backend isn’t transparent to QA testers.

    Low-code capabilities for API testing could help tear down silos inherent in the development process, enabling more people to participate in testing integrations. Belcher also stresses the importance of improving the usability of API testing for developers and non-developers alike. “We need to integrate with tools developers are already using for verification,” he added.

  • How to Mitigate Low-Code Security Risks

    How to Mitigate Low-Code Security Risks

    Gartner predicts that by the end of 2025, over 65% of development projects will use low-code builders. The field of low-code continues to expand. But what security implications does low-code introduce?

    Low-code refers to tools that enable application construction using visual programming models. Adopting drag-and-drop components instead of traditional code, no-code and low-code platforms enables non-technical folks to construct their own workflows without as much help from IT. Yet, handing power to citizen developers with less security training can be risky. Plus, low-code platforms may hold compromised propriety libraries or leverage APIs that may unknowingly expose sensitive data to the outside world. There’s also the possibility that low-code could increase shadow IT if not governed well.

    Low-code opens the door to more producers. But, does it open the backdoor, too? I recently met with Chris Wysopal, CTO and cofounder of Veracode, to discuss the growing security risk posed by low-code applications. According to Chris, hardening low-code environments involves a combination of compartmentalization, dynamic and runtime techniques, and adopting the “least privilege as possible” rule across the board. Below, we’ll expand on potential low-code risks and see what other methods Wysopal recommends organizations employ to secure the emerging low-code paradigm.

    Low-Code Security Concerns

    Verizon’s 2020 Data Breach Investigations Report (DBIR) found 43% of all breaches are connected to a vulnerability at the application layer. Thus, securing these applications, or in the case of low-code, the layer that generates them, is essential for business to function smoothly.

    When it comes to low-code, Wysopal notices a few key risk areas:

    Proprietary libraries with hidden vulnerabilities. By their nature, low-code platforms typically use a good deal of proprietary software, making securing them a bit opaque. These systems could involve a proprietary language, proprietary libraries and proprietary frameworks. If you trust a low-code platform, you place a lot of faith in the low-code vendor to have a strong security process in place.

    Some platforms do generate traditional code, which organizations can export to run on a Node.JS or Java server. Yet, this often involves a mix of code and proprietary libraries, says Wysopal. While you could run traditional static analysis on the generated code, you may miss some context.

    Lack of citizen developer security training. Citizen developers have little technical knowledge, let alone DevSecOps training. If runtime applications present vulnerabilities, expecting no-code users to mitigate them is unrealistic.

    API integrations. If low-code platforms expose an API or generate a web application via an API, the platform could be exposing sensitive data.

    Ways to Harden Low-Code Environments

    To respond to these areas, Wysopal offers some advice for securing low-code environments:

    • Perform static code analysis: Perform your own static analysis on any generated code and test for common errors. “Look at the code, and understand where it interacts with the outside,” Wysopal recommended.
    • Audit proprietary libraries: Whenever possible, Wysopal said he recommends pushing back on the vendor to question them on their application security standards and examine proprietary libraries for potential risks.
    • Verify partners: Similarly, take precautions when working with third-party partners around low-code tools. Partner tools should be carefully vetted and hold some sort of application security certification from the low-code platform.
    • Secure the API layer: Understand what APIs the application interacts with. These connections should be tested dynamically and automatically with an API scanner. Along your journey, consider these API security testing best practices.
    • Compartmentalize: Wysopal also recommends traditional isolation techniques. Consider more than just the application layer; Wysopal recommended isolating the app and running it in a virtual machine or a container.
    • Security training for citizen developers: Since citizen developers are typically not trained on appsec practices, hold security exercises to train them on fundamentals. A helpful demonstration could involve directly attacking an app to find flaws.
    • Apply the least-privilege-as-possible rule: When assigning user capabilities, whenever possible, use secure-by-default settings and allow access only to those that need it.

    By applying dynamic and runtime techniques, you may catch vulnerabilities, such as a cross-site scripting error. However, if you’re working with proprietary low-code systems, you may not be able to fix it immediately! You may have to send a request to the vendor and wait for the bug to be patched. Thankfully, most technology providers take more immediate action in response to bugs than to feature requests, Wysopal noted.

    Change the Attitude Toward Security

    On the low-code spectrum, there is the attitude that “people aren’t really doing mission-critical applications — they’re mainly doing prototyping, or back office automations,” Wysopal said. Or, since low-code applications often do not publicly expose an application, they are deemed low-risk and fall to the bottom of the security team’s inbox and list of to-dos. “We’re still in a world where people don’t secure all applications,” he said.

    However, these attitudes are a bit short-sighted, since “applications aren’t islands,” Wysopal said. Even if applications are intended for internal use only, phishing attempts could expose credentials and gain access to the network. Here, attackers could leverage sensitive resources or steal infrastructure for compute power.

    Thus, all low-code users should be aware of potential threats and change habits to address these risks accordingly. However, the onus is also on low-code vendors to sufficiently arm their systems. Having a vulnerability disclosure, bounty program and an easy means to accept bug reports from white hat security researchers will be necessary to continually patch issues.

    No-Code ≠ No-Bug

    Low-code continues to permeate more and more digital operations, opening up novel potential for citizen developers. While the low-code movement promises impressive returns, it also brings potential risk.

    To mitigate these concerns, we must level up our security understanding and evolve our approaches, Wysopal said. “Any application can have flaws and security bugs in them.” Just because you’re not writing a function in C and are relying on a visual programming model doesn’t mean you’re not introducing flaws.

    Disseminating the real risk of resources and dependencies, along with helpful consumer-facing documentation, will set the superior low-code platforms apart from the rest. Otherwise, putting faith in proprietary libraries will be a tough pill to swallow.

  • Cooperative Gaming: Software Teams’ Secret to Success?

    Cooperative Gaming: Software Teams’ Secret to Success?

    Trust falls and team backpacking trips are out — cooperative gaming is in. Don’t think embarking on group Minecraft missions could help deliver great software products? Think again. A 2019 BYU study found a 20% increase in office productivity after just 45 minutes of gaming.

    Software engineering is a mentally taxing endeavor, and many developers are under mounting pressure to deliver. To destress, many software teams use gaming to unwind during work hours. Gaming, it turns out, has added benefits of improved morale, increased productivity and lowered ego around unofficial hierarchies within the office. Setting aside time to recharge with collaborative gaming could increase a workplace’s health and help software teams work together to deliver products more efficiently.

    I recently spoke with Max de Lavenne, founder and CEO of Buildable, ally to the workplace gamer, to see why he embraces gaming as a regular part of his software developer team’s weekly regimen. Buildable adopted gaming and saw a notable impact; de Lavenne shared how other teams could similarly introduce gaming to improve performance.

    Gaming in the Workplace

    Traditional business environments don’t typically embrace fun activities on the clock — there is a clear separation between work and life. But at Buildable, gaming is integrated into the weekly routine. The Buildable engineers set aside about 45 minutes mid-week for collaborative gaming, typically on Wednesday or Thursday. The Buildable team, made up of about 20 software engineers, favors Rocket League and a modded version of Minecraft.

    According to de Lavenne, gaming is a great way to foster collaboration and intuitive group thinking. “Getting recharged is really important,” he said. He encourages the team to adopt cooperative multiplayer games; no solo play. He also discourages violent and aggressive RPGs, like first-person shooters.

    Workplace gaming isn’t unique, in and of itself — many teams at other companies will often play Haxball during coffee breaks or get into a Call of Duty: Warzone match from time to time. Yet, few leaders have the stomach to sanction gaming as integral to the developer’s workflow and overall team dynamic.

    “Engineers collaborate best when they respect each other and have confidence in each other’s skillsets,” said de Lavenne. “They are individuals, but there’s a lot of groupthink in teams.”

    Benefits of Gaming

    Gaming has proven benefits. In fact, researchers at Kansas State University found that employees who spent one or two minutes during breaks in their day playing games such as Candy Crush on their phones reported being happier than their peers. Similarly, de Lavenne noticed improved mood and other positive outcomes from collaborative gaming at Buildable, including:

    • Better team morale: Having scheduled gaming breaks to look forward to each week keeps attitudes positive. Software engineering can cause fatigue, which managers need to monitor carefully. “People get tired, and there’s only so much the brain can do,” de Lavenne said .
    • More fluid communication: Collaborative gaming focuses a team’s effort on a collective activity free from monetary incentives, bonuses or other hidden agendas. Afterward, de Lavenne said he’s noticed more fluid communication emerge among the team.
    • Increased collaboration: There’s satisfaction in doing something together, and de Lavenne reports that gaming has redefined collaboration for his team in unique ways. Employees take the cooperative, creative attitude fostered from gaming experiences and apply it to more structured, scrum-type environments.
    • Increased productivity: Dedicating one hour for gaming per week equates to about a 2% loss of labor time. However, de Lavenne reported that the productivity gains afterward surpass that lost time and more.
    • Reduction of ego: In work environments, unspoken hierarchies inevitably exist. Interestingly, collaborative gaming can completely reverse those social dynamics. Seeing senior engineers at the bottom of a gaming leaderboard can be humorous and boost confidence for younger engineers.
    • Combating isolation: Throughout the pandemic, remote work has left many of us feeling lonely. For young engineers without families, a big chunk of social interaction may occur in the workplace. Games can help maintain good spirits by bringing teams together virtually.

    Tips for Encouraging a Healthy Gaming Culture

    The overall goal is to promote enjoyable, freeform shared experiences. To enable this, de Lavenne shared a few tips on integrating games into developer team workflows.

    First off, gaming should not be mandatory, as making it a required activity would somewhat defeat the purpose. Instead, gaming should be fun and completely voluntary. If people would rather get in some physical exercise or check Instagram, they should feel just as free to do so.

    Also, as much fun as gaming with the team could be, it’s helpful having the CEO removed from this activity, de Lavenne advised. This is important so that players don’t feel they’re constantly monitored. Yet, leaders may want to have a say in the type of game, as well as when it’s played.

    Furthermore, the game doesn’t need to be an immersive RPG — it could be as simple and accessible as Jackbox. Cooperative board games like Dungeons & Dragons or Gloomhaven could have just the same effect as digital games.

    Outside of encouraging collaborative gaming, leaders can take other actions to boost morale and productivity. One such action is leaving chat channels unmonitored, so that team members can engage with each other on whatever subject they want without oversight. Another way to affect morale is location, noted de Lavenne. As I’ve written previously, hiring interest is expanding outside of traditional tech hubs. Buildable, for instance, chose to headquarter in suburban McMinnville, Oregon, for its walkability, greenery and better work-life balance than in larger cities.

    Gaming Is Where It All Began

    Multiplayer video games, LARPing, and AR/VR environments not only offer an escapist retreat — they represent nostalgia. For many programmers, video games were a doorway to the computer science field in general.

    “I’ve rarely met a software engineer that never gamed,” de Lavenne added. Gamers often seek out the highest-performing machines. And chances are, those kids with limited resources built their rigs from scratch. “This inspires you to learn about hardware, and you start to care about speed, compile and software activities,” said de Lavenne.

    From Atari to Minecraft, gaming has been a shared passion among generations of programmers. Recognizing this affinity can unite a team. Because at the end of the day, “building good software is all about people,” said de Lavenne.

    Of course, software development takes extreme diligence, planning, and structure to deliver bug-free, user-centric products. But under intense pressure to perform and meet new digital innovation demands, software developers could get burned out without a bit of distraction. With gaming, “the team really comes alive,” de Lavenne said.

    So, when things get slow, check out what’s hot on Steam. You may, ironically, just find your next productivity breakthrough.

  • How AI Will Drive Digital Twin 3.0

    How AI Will Drive Digital Twin 3.0

    Testing is a critical task for digital twins, and AI will increase the efficacy of the testing model moving forward

    Physics calculations may work perfectly well in theory. On a blackboard, academic science is pretty predictable (outside of the quantum realm, perhaps). Yet, nothing is manufactured in a complete vacuum, is it? When it comes to real-world settings, millions of factors could impact the state of a physical object — material, friction, temperature, pressure, altitude, wear… the list goes on.

    With so many tangible conditions increasing the likelihood of deviation, it can be difficult to reproduce a digital twin that accurately represents real-world conditions. This is, in part, why some believe the next generation of digital twins will be more driven by artificial intelligence (AI). By leveraging neural networks created with sensor data, production could trust intelligence gathered from trends across data points instead of solely relying on theoretical models. This could enable better predictability, optimized operational performances and overall improved products.

    To discover what the next generation of digital twins will look like, I recently met with Faustino Gomez, co-founder and CEO of NNAISENSE. Below, we’ll define what a digital twin is and why the concept needs an upgrade. We’ll also explore a few digital twin use cases to see how the next wave of AI-driven digital twins will be “learning directly from the data,” as Gomez describes.

    What Is a Digital Twin?

    No, we’re not discussing your cyber clone. Digital twins are a bit less transhuman than that. Put simply, a digital twin is a digital duplicate of a physical object or system. Digital twins are commonly used across many industries to monitor and test intricate apparatus. Industrial manufacturing, for example, often leverages a digital twin to monitor production and test the performance of equipment within a simulated environment.

    Digital Twin 1.0

    Some of the first digital twins are attributed to NASA. By reproducing mechanical objects locally, NASA could diagnose issues and test solutions for physical components 200,000 miles away in space, far beyond direct human intervention.

    The digital twin concept also found roots in other mechanical devices, such as jet engines, enabling on-the-ground mechanics to diagnose issues in the air. Using digital twins, smart factories could mimic the behaviors of manufacturing robots to test behavior and identify potential bottlenecks.

    Digital Twin 2.0 Adds More Data

    Using a digital twin, engineers could represent complex interworkings of joints and parts within a computer screen — a 3D reflection of the device. As opposed to constructing physical models, virtual simulations significantly reduced the effort and resources required to test new concepts and diagnose problems. But, simulating reality in a black box wasn’t always easy.

    Reproducing complex physical counterparts with a digital twin is wrought with complications that can be difficult to model in a standard CAD simulation, said Gomez. So, engineers began taking measurements from physical devices, such as vibrations or temperature, and integrating actual production data into the twin. This enabled operators to better forecast system performance by considering real-world elements. Yet, truly predictive forecasting was still lacking. That’s where AI comes in.

    Digital Twin 3.0 Adds AI

    Up until recently, digital twin and AI have been independent concepts. But with the advent of more sensors and greater data collection, integrating AI is becoming an inevitable evolution of the digital twin, said Gomez.

    Adding an AI layer to a digital twin would involve training a deep learning model using hundreds of time-series data outputs from sensors. Generating this data would require capturing a chronological view of how systems work functionally for some time. “Then, take data and split it,” said Gomez. “Use some to train, some to test.” The process is not too dissimilar to Walk Forward Testing, common in financial trading, added Gomez.

    Instead of using traditional statistics methods and standard physics, a neural net could unify data and find nonlinear relationships between data regardless of type. The end result could quickly identify cause-and-effect correlations between data sets (which simulated physics engines may miss) and predict future conditions based on new input.

    Use Cases

    With better intelligence, you get a more accurate depiction of possible outcomes. The use cases for crystal ball digital twins are numerous. Gomez described how 3D printing and factory maintenance could utilize digital twins:

    • Additive 3d printing: 3D printing metal powder involves a complex thermodynamic process with high-intensity lasers. The process is often less than perfect, leaving cold and hot spots from layer to layer, but ignoring them could result in a defect. By analyzing each layer’s thermal displacements, a deep learning model could be trained on the intricate details of the process. This could be used to build a more refined process model, essentially enabling manufacturers to “pre-compute the part.”
    • Predicting glass quality: Gomez described how a specialty glass company utilizes AI-driven digital twins. Their process model takes sensor readings at various chambers throughout its molten melt and cooling process, analyzing temperatures to maintain a uniform mix. Based on their controls, operators can predict what glass quality will be in the future. Essentially, it’s “a user assistance system based on a digital twin,” described Gomez.

    As you can see in the examples above, a digital twin isn’t necessarily a single object — it could represent an entire process model for a large industrial operation. According to Gomez, having this digital twin could boost testing and R&D, improve development time, and decrease time to market.

    Digital Twin 3.0 Forecasts the Future

    For digital twins to become more effective, these simulations must be faithful to current and future real-world conditions. But, computing tests for situations involving wet chemicals, unstable materials, or thermodynamics is tricky. Therefore, to Gomez, predicting such dynamics really requires “learning directly from the process model,” he said.

    The key thing here is a neural net doesn’t care what it’s monitoring. Be it torque, matter, friction, temperature, fumes or vibration, the model will learn regardless of what’s thrown at it. While the benefit here is you don’t have to know physics or chemistry, setting up this process requires much customization, which could differ significantly on the type of materials and process at hand.

    The fusion of disparate data sets could bring interesting capabilities for testing and forecasting. Yet, quality and quantity of data is paramount, noted Gomez. Furthermore, these models won’t spring up in an instant — they must be tested and validated using real-world dimensions before deemed useful. While digital twin 3.0 is a promising step forward, more research will be required to ensure the predicted digital future reflects the physical outcomes.

    AI-infused digital twins use “data from the process to predict what the process will be in the future,” said Gomez. This goes far beyond traditional 3D representations to produce more faithful, predictive digital twins.

  • How Open Source Software Powers Digital Innovation

    How Open Source Software Powers Digital Innovation

    Open source has permanently altered how we work and conduct business. Linux, and thousands of other open source projects and open standards, now dominate the software landscape, supporting emerging technologies like artificial intelligence and edge computing. Without open source, the DevOps tooling ecosystem would lack Docker, Kubernetes, service mesh, popular databases, and many CI/CD tools used by DevOps teams daily.

    Now, more than ever, companies are searching for ways to meet escalating user demands. The pandemic has amplified this urgency, accelerating existing digital transformation initiatives. Arguably, open source is the engine that now powers the digital innovation required to compete. Leveraging open source could also help check rising cloud costs. With proprietary solutions falling out of favor, many enterprise leaders are adopting open source alternatives for infrastructure modernization and application development.

    Red Hat recently released its third annual State of Enterprise Open Source report. The report surveyed 1,250 IT leaders worldwide on their open source adoption and usage patterns. The findings validate what most of us already know — open source is now fundamental to most technology stacks. Without it, most development projects would fall flat in an instant.

    Open Source Use Cases

    Open source is at the heart of modern software development. According to the report, 90% of enterprise IT leaders are using open source. So, where is open source being applied? Well, 64% of enterprises use open source software for IT infrastructure modernization. The report surmised that this could be due to Linux and open source infrastructure replacing proprietary systems.

    The report also found 54% of enterprises use open source for application development. This is no surprise, as engineers routinely adopt open source libraries and packages to construct B2B and user-facing experiences. In a digital-only environment, applications are the lifeblood for many businesses, and open source is critical scaffolding.

    The third top use case for enterprise open source is digital transformation, at 53%. This figure has increased by 11 percentage points in the last two years alone. Open source will likely continue to replace non-digital or manual processes with software-based, automated processes.

    In the enterprise, open source is used in networking, adopted by 54% of enterprises, databases, at 53% and security, at 52%. Other areas mentioned in the report include big data analytics and cloud management tools.

    In recent years, multi-cloud has become a notable trend. The idea is that organizations will seek out multiple cloud vendors to support different use cases. The Red Hat report confirmed that hypothesis, finding that 69% of respondents prefer to use multiple vendors for their cloud. As hybrid, multi-cloud setups profligate, cloud expertise and smart multi-cloud economic practices continue to gain importance.

    Containers and K8s

    Open source components are the foundation for containers and container management utilities. As reliance on containers grows, so will the dependence on open source technology. The report found just under half of enterprises now use containers in production. An additional 37% also use containers for development-only purposes.

    To orchestrate these containers, IT teams have coalesced around Kubernetes. Sixty-six percent of respondents view Kubernetes as very or extremely important. As Kubernetes becomes ubiquitous, growth in container usage is slated to continue — about one in three respondents reported their container usage will increase significantly in the next 12 months.

    “Kubernetes has become the de facto standard for Linux container development,” said Michael Hinterland, team lead, ICS cloud & automation and ICS system & middleware, Porsche Informatik.

    While Kubernetes is flourishing, there are certainly variances across industries. For example, 62% of financial services and telecommunications respondents adopted containers in production, while other sectors, such as health care and retail, saw a lower adoption of containers in production, at 47% and 50%, respectively. The highly-regulated financial services sector also scored high on Database DevOps, a separate report recently concluded.

    Open Source Benefits and Barriers

    There are numerous benefits to using enterprise open source software. For one, a commitment to open source brings transparency and collaboration improvements. Respondents also cited higher quality software, access to the latest innovations and improved security as being among the top benefits to adopting open source software.

    But open source has its drawbacks, too; 42% of tech leaders reported a lack of support as a top hurdle to enterprise open source adoption. This checks out – community-driven projects don’t typically offer traditional support systems. Without accessible frameworks for learning, creating a Center of Excellence at scale becomes difficult.

    Other top barriers to enterprise open source adoption include compatibility, code security and a lack of internal skills. Introducing open source within an enterprise means upskilling the workforce and encouraging secure-by-default settings to avoid costly misconfigurations. It could also mean working with trusted allies that further open source initiatives, the report recommended.

    Open Source Fuels Emerging Tech

    “Open source has solidified itself as an innovation engine for the software industry,” wrote Paul Cormier, president and CEO, Red Hat.

    Indeed, open source is at the heart of most cutting-edge emerging technologies. For example, 55% of respondents report using enterprise open source in edge computing and internet of things (IoT) settings. Simultaneously, 48% of respondents use open source for artificial intelligence and machine learning (AI/ML). Amid open source projects like OpenAI and GPT–3, AI is eating the software world, Forbes reported.

    Both categories are slated to increase by nearly 20 percentage points in the next two years, whereas the use of proprietary solutions for AI/ML will decrease. Open source will continue to affect buying decisions, as 83% of IT leaders are more likely to choose a vendor that has contributed to the open source community.

    Within a COVID-19-influenced world, “the problems are too big for one person, one company or one organization to solve,” said Cormier. “But it’s in moments like this where open source truly shows its power.”

  • Could No-Code Enable Everything Ops?

    Could No-Code Enable Everything Ops?

    It feels like DevOps principles are permeating every discipline, creating new buzzwords by the minute. DevSecOps. DatabaseOps. SalesOps. FinOps. MarketingOps. This “JargonOps” is clearly encouraged by marketing campaigns (and bloggers, wink, wink). Yet, the phrases do depict a real trend: all industries are getting an efficiency overhaul in the wake of increased automation.

    As I’ve covered before, low-code and no-code tools lower the barrier to entry to application development, enabling field experts to construct workflows as they see fit. For tech-savvy non-engineers, this could be a huge boon to transform copy-and-paste stopgaps into efficient workflow automations.

    I recently chatted with self-proclaimed “productivity geek” Sean Chou, CEO of Catalytic, to get his take on where this new era of automation is heading. From healthcare to logistics, advertising to supply chain management, every industry is likely to get an efficiency boost — and a lot of this digitization will be constructed using codeless environments.

    But first, how did we get here?

    From RPA to No-Code

    “The story of B2B software is largely about automation,” said Chou. If we consider classic examples of corporate applications, we see one common thread — B2B software has always sought to automate traditional office work.

    And so, it came to pass that computing and software devoured all these office tasks, from accounting to time-tracking, communications and many other areas. Though computer-based office work delivered huge efficiency boosts, these interfaces often introduced new hurdles; bad UI design, difficult navigation and a hundred open tabs, just to name a few.

    From there, robotic process automation (RPA) aimed to operationalize a growing number of tedious digital workflows across Excel spreadsheets, web apps and desktop apps. In Chou’s words, “RPA took the concepts of test automation software, and then pointed it toward production systems.”

    Though RPA’s process of recording screen interactions gathered much interest and attention (and funding), screen scraping is ultimately too fragile to be effective. It adds technical debt over legacy systems. It can be expensive to implement. Lastly, it’s not processes-centric and doesn’t implement reusable software-defined APIs.

    At that point, though, “Pandora’s box was open,” Chou said. Folks started to consider other ways to optimize, and reimagine how end users could automate workflows and stitch together interactions between disparate apps.

    Out of the ashes of RPA rose a new paradigm — low-code and no-code.

    No-Code Addresses Everything Ops

    With developer resources stretched thin and limited engineering talent available, it can be challenging to loop expert programmers into codifying traditionally non-IT processes.

    There’s not often enough developers to help with SalesOps, FinOps or any other Ops areas, explained Chou. “The only thing that can change the narrative is no-code technology that is effective.”

    Similar to how higher-level programming languages made low-level programming more accessible, Chou sees low-code and no-code unlocking accessibility for automation design by orders of magnitude.

    Four Examples of Everything Ops

    So, what are some examples of no-code use cases for automating operations? Chou outlined four specific categories that he believes will see much more citizen developer attention and action in the years to come:

    Customer Ops: “It’s really important to run customer operations as efficiently as possible,” Chou said. Especially in B2B interactions, people need to adopt various systems dictated by the customer. This can result in many ad-hoc implementations that beg for efficiency improvements.

    Financial Ops: Imagine the number of Excel spreadsheets that make up an account’s financial foundation. Consider the tedium of inputting data, generating reports or migrating calculations into other tools. There’s a ton to automate here. Providing no-code platforms to financial minds could have a “really transforming effect,” said Chou.

    People Ops: “The number of mini apps being developed for HR use cases is astounding,” Chou said. Think of all the digital tools HR must use: vacation requests, employee performance reviews, personnel databases – the list goes on. There’s a significant opportunity to clean up legacy applications within many HR departments to build a more coherent platform.

    SupplyChain Ops: Procurement and supply chain operations is another area where no-code could have an impact. Streamlining inter-company communications or engaging with potential customers are areas worthy of automation.

    No-Code? No Problem

    Across all these verticals, Chou notices three common patterns emerging.

    First, groups are using no-code to create standard operating procedures (SOP). SOPs could be used for onboarding or offboarding customers or ensuring compliance with a set of policies.

    Another everyday use is data pipelining — pulling information from many resources for analysis. A common use case is processing financial or customer data to generate a particular output. This is an area where embedded machine learning models could enrich the data.

    Lastly, a third common use case is creating workflow-powered surveys. These online forms could be used to measure employee engagement or to collect customer input. No-code workflows enable not only the collection of that data, but could allow for additional actions to be taken upon input.

    Implications and Governance

    Of course, giving non-developers the keys to the castle could have some downsides. Let’s face it — computer science is, well, a science. There’s a reason engineers are taught strict practices around version control, data governance and peer reviews. All of which must still be upheld when establishing a citizen development culture — especially at scale.

    When it comes to low-code and no-code, “governance is a huge thing,” said Chou. “Any company coming at this with no governance in mind can’t work beyond small teams.” Move beyond 10 or so citizen developers, and you’ll really need to establish some sort of shared framework or center of excellence.

    Making citizen development work in a large company requires IT visibility, data governance and a review process for new workflows, said Chou. IT will likely also want to verify that a new workflow doesn’t make permanent state changes. Otherwise, static apps that don’t touch core systems or affect persistent data stores are probably fine. “Having built-in governance is really key to making this successful,” Chou said.

    A Technology Leap

    No-code is giving more people the ability to produce unique workflows to increase their efficiency and do more with less. But with this movement comes worry. “With every technology leap, there’s always a wave of people concerned,” said Chou. Many naysayers are worried that automation will leave people out of work – “They’re coming for our office jobs!”

    Instead, Chou recommended we curb our expectations. Citizen developers are not going to build sophisticated iPhone apps any time soon. “If you think no-code is a panacea for all software, you’re coming at it all wrong,” he said. Instead, what they can actually can accomplish is far more practical.

    In reality, things are not as dire as some may make it seem. The new age of automation will likely open room for more jobs than it diminishes. “The net is always a plus,” said Chou.

  • How to Respond to Rising Cloud Costs

    How to Respond to Rising Cloud Costs

    As organizations ascend to the cloud, their cloud expenses are floating to the same lofty heights. But are all cloud costs necessary? The answer is a resounding no — there’s a lot of waste.

    In 2020, organizations wasted $17.6 billion on idle cloud resources and cloud overspending, according to ParkMyCloud. Data egress, upgrades and subscriptions to SaaS tools also crowded the cloud invoice inbox. With digital innovation a pressing concern, cloud spend is forecasted to rise significantly in the coming years. Without the right fiscal discipline in place, companies face escalating fees.

    I recently met with Eugene Khvostov, VP of product management at Apptio, to discover how organizations can rein in cloud and SaaS spending. According to Khvostov, IT departments require increased visibility, optimization and responsibility if they are to curb the cloud checkbook. Below, we’ll see what he means and discover useful tips for reducing unnecessary expenditures. Following these general guidelines could help right-size your cloud offerings across AWS, GCP, Azure and other cloud-based tools.

    Where We Are With Cloud Costs

    The pandemic has ushered in significant growth in the cloud computing and SaaS space. We’re also seeing a multi-cloud fracturing effect, as cloud service providers like GCP and Azure reach parity with AWS offerings.

    Cloud tooling grants extraordinary flexibility, elasticity and scalability. Being able to spin up or wind down an instance on a whim has permanently altered how software architecture is built. Yet, with great cloud power, comes great responsibility. “How do I make sure I’m not wasting money in the cloud?” Khvostov asked.

    According to Khvostov, as more organizations move to hybrid and decentralized multi-cloud models, more and more questions arise: What is the total cost per service? Who is spending what? Is cloud spend actually driving growth, and, if so, how much? How are new tools being procured?

    Three Cloud Financial Management Strategies

    To address these concerns and plan for cloud spending increases, Khvostov recommended three general strategies. These three phases can be thought of as research, remediation and prevention.

    1. Access and Visibility

    “You’ve got to have access and visibility,” said Khvostov.

    First, companies require access to their spending data. In this case, large, general invoices are not granular enough. You need breakdowns by service, said Khvostov, to access more fine-grained spending habits.

    Second, you need to equate those spending habits to team usage. This requires improved visibility into your organizational structure and service usage tracking at a team-by-team level.

    2. Optimize

    Next comes optimization. Ensure cloud footprints are appropriately sized for each application, and ensure you don’t have any idle resources, Khvostov said.

    Where do you have the opportunity to optimize? Well, it could be as simple as shuttering an orphaned volume, such as an Amazon Elastic Block Store (EBS), which could persist even after an EC2 instance is terminated.

    “BS volumes tend to persist unless you select the Delete on Termination,” AWS explained. So, check your configurations. Amazon provides a way to detect this using OpsCenter.

    3. Responsibility

    Once things are optimized, you need to maintain standards. “You must ensure teams drive responsibility for spend management and optimization,” said Khvostov.

    But how do you instill an optimization mindset and empower engineers to take action? Khvostov said it’s all about demonstrating a measurable return on investment (ROI) and continuing discussions between stakeholders.

    A recent report from A Cloud Guru found that 90% of organizations plan to expand their cloud services over the next one to three years, but 44% lack a plan to upskill their workforce. Most groups have their work cut out for them if they are to assign accountability for cloud costs ownership.

    Specific Cloud Costs That Bite

    So, what are some other particular areas we should be watching? Well, arguably, the most common cost is hidden data transfer fees. Surprise ingress and egress fees can be costly — take the story of Chris Short, whose personal site was charged $2700 for an unexpected 30.6 TB data transfer.

    Watch out for other cloud costs around:

    • Services automatically launching or autoscaling.
    • Idle resources due to “orphan” instances or overprovisioning.
    • Price lock on granular commitments. You could be stuck in a savings plan on an Amazon EC2 R5 instance, which would be lost upon upgrade to R6. (Khvostov acknowledged that AWS has remedied this situation, to an extent).
    • Legacy, less-efficient code can rack up cloud costs, The Information reports.

    The Cloud Economist Role Is Trending

    To unearth these costs, some believe a brand new role is required. A cloud economist is a cloud financial manager that helps spot fees, directs further cloud optimizations and calculates overall ROI from cloud investments.

    The rise of cloud financial offers is an accelerating trend, Khvostov said, one that requires the right tools and practices within an organization to become a trusted ally. “It’s definitely a legitimate role in the sense that it’s critical to organizations,” said Khvostov. “It’s specialized enough that it requires technical and financial skills.”

    Hiring a cloud economist, or building a cloud financial center of excellence, could provide a lot of value. Yet, gauging the ROI of this role isn’t immediately obvious. Some optimizations could take time to visualize, Khvostov said. A certain change could instantly invoke 20% cost savings, but you may not register that impact for 30 days or more.

    “It’s not just about cloud infrastructure — it’s about your SaaS spend, as well,” Khvostov added. He believes companies desperately need a window into what infrastructure-as-a-service (IaaS) and automation, like Chef or Jenkins, cost, along with the time savings and dollar ROI their usage equates to — because, he said, “You can only right-size what you know.”

    Trimming Cloud Spend and Cloud Costs

    Some IT folks simply fail to understand the implications of their cloud resources. Surprisingly, a recent survey by SoftwareONE revealed that 30% don’t realize that everything in the cloud is metered. To rectify this knowledge gap, organizations could use better awareness into their cloud holdings.

    The cloud is the future, and we must prepare responsibly. Yet, having fiscal discipline in a world where every software function is commodified “as-a-service” is tricky — teams require financial visibility into their cloud holdings. To rein in costs, Khvostov recommended treating it as a continuous process: access and visualize spending data, optimize your existing cloud footprint and disseminate responsibility among teams.

    Once you have a better frame of reference, you can begin to right-size cloud, right-size SaaS and appropriately right-size the team. Eventually, with the right multi-cloud and SaaS financial management strategies, perhaps that cloud bill will start to feel a little lighter.

  • Survey: The State of Database DevOps

    Survey: The State of Database DevOps

    Enterprises are undergoing rapid changes. Throughout the past year, companies have been pressed to learn a diversified set of technologies and methods to remain agile in a changing economy. In this digital-first world, data plays a heightened role; in tandem, so does the delivery of data and the persistence of the database layer itself.

    Applying DevOps principles and automation throughout the software life cycle is critical to maintaining a high level of efficiency. As with application development, databases are not immune to the DevOps takeover. Database DevOps is a discipline that considers the processes used to automate database changes and deployments — a field growing in mindshare as databases diversify.

    Seventy percent of companies now use more than one database in their stack. Many organizations now use a blend of on-premises and cloud databases. Styles are diverging, too, as non-relational databases like NoSQL gain favor, and new open-source databases emerge to fit niche circumstances. All of this means database operators may require an increased emphasis on database DevOps to handle additional complexity.

    The 2021 State of Database DevOps report, conducted by Redgate, surveyed 3,200 database professionals on the general state of their DevOps adoption. The survey also inquired about database change automation and deployment and underlying database technologies. Now, five years in, the study continues to reveal shifts in practices across industries.

    Below, we’ll review the report’s main takeaways to see how companies implement database DevOps in practice.

    DevOps Adoption Increases Across Projects

    To meet pressing digital innovation needs, DevOps has surged in importance across all industries — 74% of organizations have adopted some form of DevOps, and 26% of respondents reported applying DevOps practices across all projects — a significant uptick from previous years. The report acknowledges a slight increase in DevOps adoption among larger company sizes.

    So, which DevOps practices are companies employing the most? Well, in terms of DevOps for application development, the majority of organizations (50% or above) have adopted version control, automated builds, monitoring, automated unit tests, static code analysis and semi-automated deployments in production. Only 41% of organizations surveyed have fully automated deployments to production. These methods can be viewed as a basic benchmark for modern DevOps.

    The study demonstrated that in most organizations, DevOps is a communal undertaking. More respondents reported adopting a community of practice, rather than relying on individual decision makers, to disseminate DevOps strategies and direct planning. According to the report, a community of practice is a group of people that “makes decisions, provides recommendations, or is consulted.”

    Lack of Skills Tops DevOps Obstacles

    The pressure is on engineers to produce to meet digital transformation demands. However, organizations experiencing a developer drought may not possess the talent to introduce DevOps practices. It’s no surprise that the common obstacle hindering DevOps adoption was a “lack of appropriate skills in the team.”

    Other cultural barriers to DevOps adoption include disruption of existing workflows and a lack of alignment between teams. Interestingly, a lack of executive support was cited as the lowest hurdle to DevOps adoption, indicating universal consensus on DevOps goals. If we consider what technical barriers prevent improved software delivery and operations, 41% noted insufficient automation in release processes and 39% expressed hindrances due to legacy code.

    Continuous Delivery For Databases Increases

    Database DevOps appears to be growing in production use too. For the purpose of this study, Database DevOps is defined as “automating the delivery of database changes.” According to the report, 51% of respondents are already automating some parts of their database deployment process.

    Though growing year after year, database DevOps is nowhere near ubiquitous. Compared to DevOps in application development, database DevOps has lower overall use. For example, only 36% are implementing automated builds. And, only 22% adopt continuous delivery (CD) for all databases. Nevertheless, this could quickly change, though, as 80% of respondents are working to implement CD for database changes or have already implemented it.

    Setting up a CD process for database changes could increase overall software delivery speed. The top two drivers for adopting database automation are increasing the speed of database changes and freeing up time for other work. The third top reason, minimizing application downtime, rose in rank this year — which makes sense, as the desire to create more reliable systems is an ever-present concern.

    However, to reap these rewards, organizations must overcome some significant challenges, like synchronizing application and database changes and maintaining high speeds for database automation, among other concerns.

    Organizations Adopt Hybridized Database Approaches

    Companies increasingly use a mix of different database technologies and styles. We also see an increase in cloud and hybrid database estates.

    In terms of database styles, relational databases still make up the majority of market share — 98% of respondents have at least one relational database. However, non-relational databases, like NoSQL, are gaining popularity. Forty-seven percent of database professionals have a NoSQL database in their enterprise. Interestingly, only 2% use non-relational databases exclusively, indicating it’s a niche use case.

    Enterprises are now using more databases than ever before. Seventy percent of database professionals reported having more than one database in their stack. This figure rose nearly 10% since last year. Database types are multiplying too — 45% have more than one type of database. In terms of database management systems, Microsoft SQL Server holds the most use, at an 82% usage rate. Others like MySQL, Oracle, PostgreSQL, and MongoDB trail behind.

    Enterprises are also increasing their use of cloud and hybrid database deployment models. Eighty percent of respondents host a cloud database, and 65% report a mix of cloud and on-premises estate.

    One possible explanation for the diversified database effect is an increase in public cloud provider usage, combined with the rise of multicloud strategies. Database options are increasing as well — many free, open-source databases can cater to unique project needs.

    This “polygot persistence” could also simply be a natural occurrence within large enterprises supporting many application nuances. As Martin Fowler describes, “… any decent sized enterprise will have a variety of different data storage technologies for different kinds of data.”

    Quality Database Ops = Quality Software Performance

    In summary, non-relational databases are gaining steam, cloud and hybrid database models are becoming more common, and large organizations are increasingly using multiple database types. The landscape is forecasted to diversify further, meaning database DevOps is stepping up to manage this complexity.

    The report indicates a correlation between quality database Ops and overall software quality, noting that organizations that release more frequently typically deploy database changes in much less time. To achieve these goals, “… the next area for database DevOps is in the data engineering and analytics space,” writes Pramod Sadalage, director, data & DevOps, ThoughtWorks.

    While it’s important to take a vendor-endorsed study with a grain of salt, it’s safe to say we will likely see more interest in database DevOps, especially as DevOps themes continue to permeate through the entire spectrum of software development and delivery.

    For more information, you can download The State of Database DevOps report behind an email gate here.

  • Dual-Track and Low-Code’s Excellent Adventure

    Dual-Track and Low-Code’s Excellent Adventure

    Digital transformation can be a bogus journey. Here’s how low-code and dual-track transformation efforts can help.

    Large-scale enterprise digital transformation can take a while. Transformation initiatives like cloud migration, mainframe upgrades or refactoring monolithic systems could easily take five to seven years. Yet, this slow pace is at direct odds with today’s digital innovation standards and the pressure to release software rapidly.

    COVID–19 has amplified this drive to innovate, especially complicating matters for non-cloud-native companies, which struggle to attract and retain talent amid a developer shortage. Not to mention, developers are experiencing surmounting pressure to perform, which could result in burnout if not supplemented by new strategies.

    To maintain agility while diligently advancing multi-year digital transformation projects, some proponents argue for a dual-track approach. This approach uses low-code platforms to increase operational agility (the fast track) while digital transformation initiatives (the slow track) progress methodically on the side.

    I recently sat down with Jay Jamison, chief product and technology officer at Quickbase, to discuss today’s digital transformation concerns. According to Jamison, implementing a dual-track approach affords great rewards, but it requires an organizational cultural shift. He recommends establishing centers of excellence (COE) to evangelize low-code practices and encourage citizen development, enabling subject matter experts of lower technical acuity to automate operational workflows and operations.

    Dual-Track Transformation

    “Digital transformation is an important element for businesses of all sizes,” said Jamison. However, mainstream digital transformation projects, like rationalizing an ERP, moving to the cloud, or upgrading a network, can take an exorbitant amount of time and money. It’s hard for IT to meet business requirements. “We really need a second track of digital transformation — one that is more agile.”

    Jamison likens dual-track transformation to chipping out “small rock” projects from the larger digital transformation cliff. This strategy focuses on streamlining realizable goals in shorter timespans, empowering teams of people — with subject matter expertise — with tools to transform and digitize their workflows.

    Low-Code’s Role in Dual-Track

    A recent study conducted by Harvard Business Review found that 80% of executives “strongly agree” that a dual-track approach is essential for digital transformation. Yet, only 31% of companies have adopted said approach.

    To realize a dual-track process, Jamison recommends low-code for its “low-risk, high-throughput capability.” According to Jamison, low-code enables the edge of business to quickly adapt new solutions and run them in parallel with big rock digital transformation. For example, non-engineer government employees in a Texas county recently used a low-code platform to help distribute coronavirus vaccines.

    For a citizen developer culture to flourish, “you really need to have all hands in the pile,” Jamison said. IT stakeholders and business members must collaborate and work together to bring low-code to fruition. This process combines setting goals, outlining guardrails, and forming learning groups with progressive leadership at the helm.

    Introducing Centers of Excellence

    Naturally, many factors can affect IT innovation (leadership commitment, collaboration, organizational expectations, etc.). But arguably, the most common roadblock to digital innovation is culture. In fact, 45% of executives report that organizational structure is a major transformation challenge.

    That’s where centers of excellence (COE) come in. No, centers of excellence is not an archive of Bill & Ted’s adventures. COEs aim to fix the cultural barrier by encouraging progressive leadership, collaboration, and innovation. The Playbook for Centers of Excellence defines COE as “a concentration of expertise and resources in a specific capacity, designed to maximize the focus on a specific project.”

    A COE aims to foster shared understanding between IT leadership and business leadership. In Jamison’s words, “a center of excellence provides the ability to utilize low-code to sustainably build and empower applications in a safe, sustainable way.”

    Building a Most Triumphant Center of Excellence

    The COE goal is to empower business users to innovate. Yet, COE is a governance framework, which seems a bit contradictory. So, how does an organization go about implementing it? Well, there is a range of how companies adopt COE, but Jamison shared some common steps:

    Step one is establishing your goals. This could be increasing productivity, enhancing customer satisfaction or improving service reliability. Step two involves inviting the right people and perspectives — the key personnel to own the COE program.

    Next, develop transparent policies around security. This helps preempt concerns around data security and compliance. Finally, now that goals and rules are established, communicate the initiative across an organization.

    Throughout this process, investing in training is paramount. There are many project management philosophies for encouraging low-code adoption, but Quickbase recommends a training and education team to “roll out information and goals in a structured, targeted way”. They also encourage a community focus — user groups, internal events and knowledge-sharing initiatives can build excitement within the employee base.

    Benefits of Low-Code Enabled COE

    A COE aims to bring disparate parties to the same table. For example, companies could hold what Jamison described as an “appathon,” in which business executives present their goals, like rolling out new installations or driving 5G deployments. Then, priorities are assigned to individual groups, who apply domain knowledge to construct applications.

    Setting up a COE by accelerating ground swell innovation “catalyzes the dual-track innovation that we think is really key,” said Jamison, reducing the mean cycle-time to introducing new features. Low-code could also lower the integration barrier, enabling connectivity with CRM systems, or connecting with big, heavy systems like SAP or Oracle.

    Jamison also described low-code platforms as encompassing “the spirit of microservices,” as they introduce a relatively lightweight application process and easy deployment model. Furthermore, if teams are tapped into the same PaaS, they receive data reusability benefits, he said. “Everyone can utilize the same data in a way that’s relevant for what they need.”

    Citizen Developers: Bogus or Non-Bogus?

    Personally, I think it would be great if the future of application development ushered in a democratizing effect, allowing more people to participate in software creation. Yet, the skepticism around a fully working citizen developer model is hard to shake.

    The “citizen developer” seems to have a negative connotation among coder elites, too. As one reader commented:

    “So top talents are overpaid extravagance and bored secretaries working from home will be driving innovation by writing no-code workflows, am I right with understanding what is written here?”

    I pressed Jamison on the reality of the citizen developer concept in practice and how he responds to naysayers. He jokingly equates it to a witch hunt.

    “Saying that only people that get to participate in software development is professional software developers is ridiculous,” he said, noting the importance of subject matter expertise.

    According to Jamison, companies often fail when they attempt to transmit subject matter expertise to developers. Organizations would rather teach subject matter experts low-code than teach the subject matter to programmers, he noted. It’s not hard to imagine why allowing subject matter experts to build their own workflows for specialized fields, such as accounting, natural science, or mathematics, makes sense.

    Regardless, Jamison adds there’s nothing really to worry about — “The full stack developer isn’t going anywhere.”

    Dual-Track Digital Transformation? Excellent!

    A dual-track approach combines large-scale transformation and rapid-cycle development. By separating “small rock digital transformation” out from the larger puzzle, organizations can increase operational agility to meet business needs a bit quicker.

    Establishing a center of excellence seems to boil down to bringing people together to share ideas, collaborate, and drive better productivity. If anything, this way of thinking will help operationally-intensive industries consider how they can automate parts of their business.

    Ultimately, the goal here is to enable organizations to collaborate better and drive better productivity. By empowering more creators, you can start seeing creativity at the edge of business emerge too, said Jamison. If that involves citizen developers creating event-driven workflow automation, so be it. “We invite people to take on the challenge,” said Jamison.

    Hopefully, then, business and IT can be excellent to each other.