Category: Enterprise DevOps

  • Are ERP-Centric Enterprises Being Left Behind?

    Are ERP-Centric Enterprises Being Left Behind?

    Many large enterprise companies are rife with departments that depend on customized ERP software applications. These systems accommodate vital business tasks ranging from inventory management to ordering capabilities to accounting programs to CRM platforms—all of which were custom-built to fit the enterprise’s specific needs and meet the KPIs of each unique organization. These applications tend to sustain grades, patches and reworks, sometimes resulting in multiple redesigns over the course of decades, and these often are not properly documented. To make matters worse, these ERP software systems also tend to be written in outdated programming languages such as ABAP for SAP and PL/SQL for Oracle, making them a challenge to update.

    For example, 91% of respondents in a recent ASUG survey of 110 SAP companies said they use custom code, and 41% of those applications were created at least six years ago. The majority of these companies said that the existence of these custom code environments has become a hindrance to digital transformation and is holding back innovation.

    These conditions create the bloated and overly complex legacy environments that are now commonplace in the enterprise market. Widespread, customized core business systems have made digital transformation particularly daunting for enterprise companies. They require significant developer resources, excessive time commitment and ample expense to strip down and redesign these custom systems for the cloud. 

    A Lack of Developers—and of Developer Interest

    U.S. Department of Labor statistics from December 2020 indicated that there will be a shortage of 40 million skilled workers worldwide, amplifying these issues further. That figure is predicted to grow to 85.2 million by 2030. What’s more, new developers entering the marketplace gravitate to more contemporary programming languages such as JavaScript and Python, as opposed to wanting to revamp archaic code used in most ERP solutions, such as ABAP.

    For this reason, large enterprise companies in a range of verticals are lagging behind when it comes to upgrading mission-critical software applications to the cloud or to a more modern architecture. These organizations are well aware they stand to gain significant productivity benefits from digitally transforming their systems so that, for example, inventory data can be accessed via a mobile device or shared from anywhere on a cloud-based platform. Yet these large organizations are stalled in their efforts to modernize. Some companies have gone as far as to scrap their current systems and start reprogramming their software applications from scratch—a hugely expensive, lengthy and laborious endeavor that’s also full of risk. 

    Leveraging Low-Code to Simplify Enterprise IT Environments

    One method that has shown progress in unraveling these overwrought environments is to apply automation through low-code integration. Ideally, an effective low-code development solution will automatically convert existing business processes to modern, cloud-native applications, allowing the organization to maintain the business logic it has built into those applications over the years, while drastically reducing the time commitment, costs and potential disruption associated with traditional, long-form integration. In SAP environments, for example, using a low-code rapid application development solution can separate the application layer from the core SAP system, automating a large portion of the migration while preserving a “clean” SAP core.

    With many organizations under pressure to keep budgets under control in what continues to be an uncertain economy, these large-scale companies are looking for ways to modernize their operations on a more controlled schedule. When enterprises use low-code solutions—and furthermore, don’t have to search for developers of these more archaic languages—they’re free to approach digital transformation in a far more incremental and cost-effective way.

    For instance, the 12 million users of SAP’s solutions are being compelled to upgrade to the modernized version of its product, S/4HANA. Yet the American SAP User Group (ASUG) research shows that only 33% of these organizations have taken on the challenge so far despite the looming end-of-life of the current prevalent version. This is because lacking a low-code development solution, companies are being pressured to convert and rebuild all their business processes in one fell swoop. Few organizations have the resources to absorb such a drastic and risk-prone undertaking. 

    A rapid application development program acts as a bridge for companies that are looking to modernize their businesses, allowing them to upgrade their applications incrementally, according to their own schedules. Low-code can be a catalyst for technical debt-ridden enterprises, allowing them to simplify complicated legacy environments and get down to the business of innovating their systems.

  • But What About When You Fail?

    But What About When You Fail?

    One thing Agile and DevOps definitely brought IT was a more accepting view of the whole “mistakes happen” mantra. Crashing systems is no longer a guarantee of a free ticket out the door. Indeed, off the top of my head, I can think of at least two cases where a CIO themselves checked in sloppy code and it trashed the system—CIOs at orgs big enough that they probably should not have been coding in the first place.

    And that brings us to today’s blog. The standard response to “But what if there is a bad update?” is “We’ll just roll out a new version!” That works in some instances. In a lot of instances—most that involve GitOps as part of DevOps—it doesn’t. Implosions can be huge and take out chunks of infrastructure. So you need a better plan than just assuming you can fix it with a forward update. Do you have rollback capability? Are you making use of it, assuring all is set to roll back if needed? Testing to make sure rollback works with the other changes made across the system in this update?

    That is part of the problem. Fixing with a new update is the best option, simply because in the age of massively distributed, microservices-based solutions, rolling back comes with a ton of baggage—enough that it may not be viable for you. Okay. So you can’t quickly roll forward. You can’t quickly roll back. Quick! What do you do?!

    And I don’t have the answer to that question, because I’m not in your organization working on your systems. At some point, all IT is personal. And this is one of those points. You need to know what your best options are if an update spirals everything, and you need to have a plan to implement it. But what your best options are is not going to be the same as the next org. This is that point.

    One option is to keep the ability to build the entire system. That’s a large setup, but every minute systems are down is hurting the company. And that’s what you need to plan for. Think of it as disaster planning for DevOps. A man-made disaster that destroys systems and infrastructure.

    So, like any other disaster planning scenario, walk through the chain that makes the system work, identify weaknesses and list ways to address them. Then test those to make certain they do what you hope they will. Then, set up an automated system to keep this whole plan up-to-date.

    In short, we’re in a new automated landscape, and we need a new automated tool to pull our rears out of the fire when the inevitable happens. Be it from dev error or malicious attacker, it is a safe bet that, sooner or later, you will have a massive systems outage that your DevOps toolchain can’t adequately address. Know what you are going to do. Or at least have thought about it, so you’re not just starting to think it through while coworkers and customers can’t access systems.

    And keep rocking it. This is just another layer of protection for all the hard work you’re doing. Take the extra step. Like insurance, if you ever need it, you will absolutely be glad you did.

  • Global Next-Generation Software Engineering Conference

    Global Next-Generation Software Engineering Conference

    GNGSEC sounds the clarion call for software engineering change!

    The pandemic made 2020 the year where we see some of this decade’s most exciting digital technologies become commercially viable, and others finally go mainstream. Software is the core of digital transformation. In the digital era, the methods by which we produce software also need to be digitized. In the software engineering field, the adoption of AI, big data, cloud computing and cloud-native are accelerating the digitization for businesses.

    Global Next Generation Software Engineering Conference (GNSEC) focuses on a full range of software engineering technologies and is committed to defining the standard of the next generation of software engineering. The event will gather 1000+ Attendees and it will be live with virtual experience where senior software engineers, architects, and team leads connect, learn, and grow. Experts and developers from various backgrounds will be sharing their cutting-edge technical practical experience and cases.

    The majority of the attendees of this event will be from the Southeast Asia region (Singapore, Thailand, Indonesia, Philippines, Malaysia, Vietnam), India and China. Due to the current travel restriction, GNSEC 2021 will be virtually presented via Airmeet. It will offer attendees with virtual exhibitions, virtual conference, peer networking session, business matching session, virtual discussion group.

    Global Next Generation Software Engineering Conference (GNSEC) is co-presented by DAOPS Foundation and Escom Events. The event is part of digital innovation series events that features AI, DevOps, Cybersecurity and digital transformation. Previously the event was presented in Shanghai, Shenzhen, Beijing, California and Hong Kong.

    There will be more than 30 speakers from DBS Bank, ING, IBM, Daimler, Tencent, JD.COM, DevOps Institute, Gitlab, New Relic, SUSE and more to share ideas, experiences and case studies in software engineering.

    Register here to join​​ the event with us to rebuild software engineering.

  • Tips for a Successful DevSecOps Life Cycle

    Tips for a Successful DevSecOps Life Cycle

    A DevOps implementation, if done correctly, can do wonders for any organization that’s on the hunt for efficiency, productivity and speed. As per the 2020 survey conducted by Atlassian, 99% of survey respondents said that DevOps has had a positive impact on their organization.

    However, it won’t do any good if security isn’t prioritized while adopting the DevOps best practices. That’s where DevSecOps comes into play.

    Enterprises need to understand that leveraging the alignment of development and operations for workflow enhancement while ignoring underlying security issues is like trying to generate electricity from a broken solar cell on a cloudy night. Undoubtedly, the success of any DevOps initiative predominantly relies on general security practices and a robust security mechanism to mitigate risks—this is where DevSecOps plays a vital role.

    There are several essential components of a successful DevSecOps implementation. Here’s how you can leverage them in your organization.

    The Emerging Need for DevSecOps

    DevOps can be simply defined as the next evolution of an agile development environment that bridges the gap between operations and development teams. Enterprises increasingly are turning to DevOps to boost their application development speeds and operational efficiency. Often, though, prioritizing development velocity means security concerns are left behind.

    That’s why there’s an equally growing need for a secure, efficient yet still agile development environment. DevSecOps puts security at the forefront of the entire operational and development process, which safeguards good cybersecurity hygiene right from the beginning. Implementing DevSecOps directly benefits developers and operations teams working hard to deliver a flawless application in a short period of time. Moreover, this approach encourages enterprises to develop secure code and thus, secure applications. To do so, there are several resources and strategies to help implement DevSecOps into an organization’s software development life cycle (SDLC).

    Secure Coding Practices

    Secure coding is perhaps the most crucial aspect of ensuring stringent security right from the beginning of the SDLC. Developers who aren’t following secure coding best practices, security guidelines and maintaining compliance while coding the application are inviting many security risks, including breaches of sensitive data and exploitation of an individual’s identity.

    Hence, an organization must focus on enhancing the skills of developers and ensuring that they are following security best practices while their application is in the initial stages of development. Moreover, establishing and adherence to coding standards are also crucial, as they help application developers to write clean code.

    Leveraging Automation

    As we know, automation is a driving force and a key aspect in DevOps; the same goes for DevSecOps. Automating security is vital to keep up with the pace of application code delivery, especially for more prominent organizations working on various code versions. Choosing the right automation testing tools can be beneficial for organizations that wish to speed up their operations and enhance the reliability of code when it comes to security. Organizations should strongly consider using at least static application security testing (SAST) tools that help developers identify potential vulnerabilities in code in the early stages of the development life cycle.

    Early Stage Testing

    The biggest mistake for any organization leveraging DevSecOps is testing the application only when it’s finally completed. Beginning early with automated testing offers numerous advantages, including identifying and resolving vulnerabilities in early development stages, a quick and inexpensive way to fix issues and saving valuable time during deployment.

    However, testing during the early stages of the SDLC may also halt the overall DevOps development pipeline, so take that into consideration from a security perspective.

    Implementing DevSecOps

    Implementing DevSecOps isn’t a quick process and requires specific steps to get it right.

    Planning and Development

    A solid planning strategy for successful implementation must include test criteria, design, vulnerability models and analysis. Once the planning is complete, the next step is development. Development teams should initially evaluate the overall complexity and challenges of their current practices. Adding a code review system in the development process can help to encourage safe and reliable coding practices in a DevSecOps environment.

    Build and Test

    Now it’s time to build! You can use automated tools to help combine source code with machine code; many of those build automation tools offer security features like automatic vulnerability detection. Once the build is complete, rigorous testing is carried out through various automated testing frameworks that ensure the highest level of performance and security.

    Monitoring

    Monitoring is another crucial aspect of DevSecOps implementation to ensure your security mechanisms are performing up to snuff.

    Undoubtedly, DevSecOps has revolutionized the way enterprises handle security, especially in challenging DevOps environments. Businesses that want to get to the next level of agility without sacrificing robust security should be implementing DevSecOps.

  • The Role of APIs in Mainframe Modernization

    The Role of APIs in Mainframe Modernization

    Mainframes are historically clunky systems. They are incredibly secure and can process impressive calculations, yet they are complex to manage and challenging to extract data from. This reality is at odds with modern cloud-native infrastructure, which emphasizes distributed computing and portable microservices. So, how can existing mainframes adapt to this connected, data-driven economy?

    I recently met with Dr. Alex Heublein, president at Adaptigent, to talk about the state of legacy modernization. According to Heublein, many mainframes are here to stay for the foreseeable future. However, enterprises are continuing to get value from these systems by adding an integration layer on top of them. By using a REST API layer on top of a mainframe, enterprises could finally open up these systems. This could facilitate bidirectional communication with newer cloud-native services, enable phase migration and edge closer to real-time processing.

    State of Mainframe Modernization

    The software industry changes every day. “Nothing in the tech field is permanent,” said Heublein, yet some legacy technologies will persist into the foreseeable future. In terms of mainframe adoption, Heublein sees a couple of categories of users. First are the large financial services organizations, such as large banks and insurance companies, which are considerably invested in their mainframe architectures and which will likely continue to rely on mainframe environments for their policy management and core infrastructure. “I think there’s a lot of good reasons for that,” explained Heublein. “If you want to process billions of transactions a day, [mainframe] is still a good option.”

    That said, regulations and market pressures are influencing a global movement toward open banking. For example, PSD2 mandates open banking in Europe and industry consortiums like Financial Data Exchange (FDX) are directing it in the U.S. market. These initiatives are encouraging financial services to transform their legacy environments and spurring cloud migration, especially for applications that aren’t mission-critical.

    Heublein pointed out a few different patterns in enterprise mainframe modernization. One approach is that enterprises are rewriting or re-architecting old COBOL applications and keeping them on-premises. Or they are porting and re-platforming the application; moving a COBOL app to a Windows or Linux system, for example. Finally, some are shifting straight to the cloud. For solutions that don’t need to be on the mainframe, enterprises may shift toward cloud-native to free up capacity.

    APIs Enable Legacy Modernization

    Distributed cloud technology is helping businesses scale to the right level. Yet, it’s difficult to jump off the mainframe entirely, noted Heublein. “Very rarely can you take all your stuff and move it all at once,” he said. “It’s a tough pill to swallow.” It’s especially difficult to move core banking infrastructure to a distributed environment—an instant shift could introduce security risks. For this reason, he recommended a ‘phase migration’ approach in which individual components are slowly migrated. For example, an insurance provider may shift their claims engine, policy engine and ratings engine to the cloud one at a time.

    As a result of phase migration, however, engineers must simultaneously support both on-premises and cloud-based applications. They must also ensure bidirectional communication so that these systems can exchange data. This is where a REST API transactional layer is necessary, Heublein explained. Such a layer could enable bidirectional communication between the mainframe and cloud-native applications, abstracting complexity when calling out to the modern world, or intercepting cloud data and filtering it into a format mainframe technologies can process.

    To increase developer usability, abstracting complexity from the mainframe integration process will be the key to building out connectivity. This could involve a codeless development approach with a COBOL sub-routine to call external systems, Heublein said. Automatic restructuring of data will be necessary to enable fluent bidirectional communication, too.

    Four Business Drivers of Mainframe Integration

    So what is the end result of putting an integration layer on top of mainframe systems? According to Heublein, there are four key business benefits of using such an abstraction layer:

    1. Orchestration: Software ecosystems often need to make multiple calls to different applications to collect and transform data. Yet, mainframe environments are very complex and challenging to integrate with. More usable integration fabric for mainframe systems could enable better orchestration of workflows in a developer-friendly manner. By using a REST interface, all apps could be calling the same interface, thus greatly simplifying orchestration between disparate services.
    2. Sustained Adaptability: The world is rapidly accelerating, and enterprises need to maintain and modify these APIs very quickly. Organizations require “sustained adaptability” to adapt and survive in the digital world, said Heublein. Mainframe systems, however, were not designed for the 21st century. Thus, low-code integration layers are critical to empowering rapid development.
    3. Data Transformation: Data is powerful for the business. Data is not only a commodity in and of itself but also is required for AI and machine learning. However, mainframes have a completely different way of encoding data. Thus, there is a need to transform this data into a modern format that businesses can consume. An integration layer between these systems can deliver an improved data consumption model.
    4. Real-Time Processing: You may think online payments happen in real-time; however, payment apps like Venmo are really just an abstraction layer over bank transactions, which may clear as little as twice a day. In short, mainframes are not great at real-time processing. However, more usable integrations to this infrastructure could enable user-facing applications to interact more closely to real-time.

    Another concern is security. Opening up mainframe environments that house core financial infrastructure must be done with extreme care—especially as API attacks remain the number-one rising threat in the software landscape. All organizations building out new integrations will require a secure layer from an outbound standpoint. Heublein said this often involves placing an API management layer or gateway in front of the integration layer. This system enables authorization and functions like a firewall between the internal and external world. It can also enable logging and prevent denial of service attacks, he explained.

    Improving Agility for Legacy Systems

    Legacy systems are entrenched in older technology and carry lots of technical debt, forcing operations to move at a slower pace. Maintainers are also culturally a bit different than the newer breed of cloud-native engineers. “Mainframe people are not … fast,” said Heublein.

    DevOps folks want to build, test and promote projects to production very quickly and, arguably, this agility is necessary to satisfy rising digital innovation demands. Improving integration capabilities for on-premises systems could enable businesses to better utilize preexisting investments while also improving agility.

    Low-code/no-code capabilities could also increase collaboration potential, Heublein added, which could bridge the gap between on-premises engineers and fast-moving business folks. When adopting low-code/no-code tools for mainframe integration, he cautioned against general-purpose citizen developer frameworks. Whereas universal integration layers can work well for many situations, he stressed that a very specific boundary box is best for building APIs for mission-critical applications.

    How is your organization approaching modernization for legacy systems? Let us know in the comments below!

  • Sumo Logic Extends Observability Scope and Reach

    Sumo Logic Extends Observability Scope and Reach

    During the online Illuminate user conference, Sumo Logic revealed how it is integrating capabilities it gained with the June 2021 acquisition of Sensu, a provider of open source monitoring tools, with its continuous intelligence platform in addition to adding support for data collected via open source OpenTelemetry software.

    Sensu Go, a set of tools for implementing monitoring-as-code, is now a core part of the Continuous Intelligence Platform. The company is also now making available in beta Sumo Logic OpenTelemetry Distro and Ecosystem Support, a single agent through which all logs, metrics and traces can be collected from instrumented applications. Sumo Logic is also supports the AWS OpenTelemetry Distro, an offering from AWS that provides similar capabilities. Sumo Logic has also increased the capabilities of Sumo Logic Free to include monitoring-as-code tools created Sensu.

    Bruno Kurtic, vice president of product and strategy, said as responsibility for managing applications shifts left toward developers it’s critical to provide observability capabilities that can be programmatically infused into applications. As part of that effort, Sumo Logic is now also offering Sensu Plus, which provides access to an analytics engine for the data the platform collects.

    Sumo Logic is also adding machine learning algorithms to provide more context to the alerts the platform generated via an Alert Response feature that is part of the core platform.

    The company is also adding support for the operator software that Red Hat created to make it simple to deploy Sumo Logic Continuous Intelligence Platform alongside the Red Hat OpenShift platform for building and deploying applications on top of Kubernetes clusters. In addition, Sumo Logic is also providing integration with a total of 38 either new or updated observability applications that integrate with a wide range of third-party platforms.

    Finally, Sumo Logic is extending the security observability platform it provides based on its platform to add support for containers and serverless computing functions alongside virtual machines and integrations with third-party endpoint detection and response (EDR), network detection and response (NDR), threat intelligence and managed detection and response (MDR) platforms. The company has also added or updated 26 applications for specific third-party platforms.

    Kurtic said Sumo Logic makes it possible for developers and IT operations teams alike to observe an IT environment on an end-to-end basis via a single pane of glass. In some cases, the members of an IT operations team will require a user interface to manage that environment. Developers, meanwhile, might prefer an application programming interface (API) through which observability data is made accessible from within a continuous integration/continuous delivery (CI/CD) platform.

    Regardless of the type of user, however, IT teams should not have to hire their own data science team to analyze observability data. Instead, the observability platform should embed machine learning algorithms that surface all the benefits of data science without the need to hire additional headcount, he said.

    One way or another, DevOps teams will soon routinely benefit from the additional levels of context that observability platforms provide. The challenge will then be determining how best to fix a raft of issues that many DevOps teams today simply didn’t know existed.

  • The New(ish) Complexity

    The API economy and microservices architectures, led by Agile and DevOps, have introduced a massive amount of hidden complexity. I call it hidden complexity because we have a tendency in IT to focus on the problems in front of us, resolve them, then focus on the new problem in front of us. This leaves technical debt in little piles behind us, like the bits that fall off the ends of a snowplow blade (or the asphalt that squishes out the end of the steamroller for our southern friends).

    I was thinking about this when I was looking into how a majority of you handle scalability. The layers of complexity for scalability are surprising. In the past, we invested upfront to be prepared to scale to the levels we need. Today we allow expenses to grow as users grow. That is a great model if users are tied to revenue. If there is any fungibility at all in the correlation of users to revenue (and there almost always is), this is a potential disaster waiting to happen. “Our traffic spiked 200%, but revenue only grew 20%,” is pretty much the last thing your business leaders want to hear. Because in the “Pay as you grow” world, a 200% spike in users means a significant spike in overhead. How much overhead varies pretty wildly, of course, so I’ll just say “Guaranteed and significant” and leave it at that.

    This is a known bit of complexity, but those of you I’ve spoken with have done little to hedge against it. That’s … interesting. How many other hidden land mines are out there, being ignored by us?

    What can you do? That greatly depends upon your business. with internal systems, you have a lot of say in usage. External systems are where this is potentially an issue. You can pay your provider to block DDoS, so at least you’re not paying for that bandwidth if that happens—and most of the people I spoke with really do have some form of DDoS protection that stops traffic before it becomes a fee-generating hit. But for actual growth, the best you can do is get with business leaders and determine what influences the correlation between users and revenue. What makes those people upping your traffic actually spend money while they’re upping your traffic? What can you do to convince more of them to spend money? What services are you not offering that might be useful to those customers and not significantly increase overhead. Basically, this is the modern equivalent of the conversion problem. They’re on your site, how do you convert them to paying users of your site.

    We really can’t get more detailed than that—you know your business, and a media site is a different model than a retail site, for example—so you’ll have to develop beyond that. But have a plan that does what you can to make sure that when expenses go up, revenue goes with it.

    And keep rocking it. We’re outside the historical IT realm here—because you’re all rocking it so well that we have new things to worry about. Give those customers rockstar apps to work with and figure out how to make them paying customers. Develop those bonds with business leaders; you do indeed need each other.

  • THE UPSKILLING IT 2022 SURVEY IS NOW OPEN

    THE UPSKILLING IT 2022 SURVEY IS NOW OPEN

    IT SKILLS IN FOCUS

    By Eveline Oehrlich, DevOps Institute

    According to McKinsey, nine out of 10 global executives and leaders are saying that their organizations are facing a skill gap or expect to develop additional skill gaps in the next three to five years. During the past few years, IT organizations, in particular, have already been facing skill gaps. The desire to accelerate technical innovations and digital transformation only makes it harder to identify and narrow current and future skill gaps. IDC predicts 75% of organizations will have a comprehensive digital experience implementation roadmap by 2023, up from 27% today. While 2023 is still a few years away, it is essential that IT organizations prepare for what skills are necessary, how to manage their talent supply and look at the underlying skills of people that must support this digital journey.

    DevOps Institute research team and sponsors, Palo Alto Network, Cycloid, and The Linux Foundation, are excited to announce the launch of the Upskilling IT 2022 Survey. The survey takes approximately 25 minutes to complete, and we’d love to hear from you. Your answers will allow us to better understand which IT skills are essential for the future, ensuring that organizations are able to accelerate their digital journey.

    The Upskilling IT 2022 Survey, by DevOps Institute with its yearly sponsors, is the only and longest running research of its kind. The results provide an unbiased view into what skills are essential across a variety of IT and other skill domains.

    Like the past three research reports, our goal following this year’s survey is to share a detailed analysis to help various leaders and roles, in order to help them determine what skills they should hire, retrain or upskill for, following the publication of the Upskilling IT 2022 Report in the first quarter of 2022.

    Don’t let IT Skill gaps be a competitive liability for today or in the future

    The survey is open for everyone, regardless of the size of your organization, or your organization’s industry. This year we have expanded the focus beyond DevOps and are homing in on cognitive and human skills.

    Some of the key topics we look to deep dive into this year include:

    1. Exploring where the biggest skill gaps currently exist and within which roles, such as leaders, developers or operations.
    2. How skill gaps are identified across the IT industry.
    3. How upskilling is done, funded and executed.

    While each person has their own unique experience, expertise, examples of execution or outcomes and various tolerances for exploration, for the fourth year, we are setting out to explore what the must-have skills are within different skill domains. This year we are focusing on the skills needed within modern IT organizations today and in the future.

    There are a variety of roles that are involved in recruiting, hiring or working within IT organizations. We are setting out to highlight and understand which must-have, nice-to-have and optional skills are essential within key skill domains. The skill domains include Automation, Human skills, Technical skills, Functional skills and knowledge, Process skills and framework knowledge, Business skills and leadership skills. We are targeting the community of IT leaders, developers, practitioners, team leaders, consultants, human resources and other individuals who are familiar with modern IT best practices, such as DevOps. We are targeting all industry verticals and all company sizes. Our survey questions are developed with input from a variety of team members and experts, and our goal is to achieve a sample size of 4,000 individual responses across North America, Europe, Asia Pacific and Latin America.

    Help us shape the future of IT and DevOps and make your voice heard by completing the survey now.

    The survey will remain open until January 31, 2022. We look forward to hearing from you.

    Take the survey now:  https://www.surveymonkey.com/r/Upskilling2022Blog

  • LogDNA Previews Streaming Module for Observability Platform

    LogDNA Previews Streaming Module for Observability Platform

    LogDNA is making available on a limited basis the ability to process streaming data to its observability platform as part of an effort to give DevOps teams more control over log data.

    Tucker Callaway, LogDNA CEO, said the data ingestion pipeline that LogDNA already provides can ingest, parse and normalize massive amounts of structured and unstructured log data. LogDNA Streaming adds the ability to automatically send log data to any application or analysis tool.

    That capability also helps reduce storage costs by limiting the number of copies of log data that might otherwise be created, he added.

    LogDNA also expects that DevOps teams will also be able to analyze data in flight as part of an effort to make it simpler to surface issues even before log data is stored, noted Callaway.

    While there are multiple streaming data platforms available, Callaway said LogDNA opted to build one on its own to meet the requirements of an observability platform. Ultimately, the goal is to shift observability left at the data level to provide more granular insights in real-time, he said.

    That capability will also lay the foundation for a wide range of autonomous processes that would be driven by the observability data collected via LogDNA, added Callaway.

    LogDNA

    Observability has always been a core tenet of any DevOps best practice. Most DevOps teams typically focus on some form of continuous monitoring to more proactively manage application environments. However, it can still take days, sometimes weeks, to discover the root cause of an issue. Monitoring tools are designed to consume predefined metrics to identify when a specific platform or application is performing within expectations. The metrics tracked generally focus on, for example, resource utilization.

    Observability combines metrics, logs and traces—a specialized form of logging—to instrument applications in a way that makes it simpler to troubleshot issues without having to rely solely on a limited set of metrics that have been pre-defined to monitor a specific process or function. DevOps teams can more easily employ queries to interrogate data in a way that makes it easier to discover the root cause of an issue. An observability platform correlates events in a way that makes it easier for analytics tools to identify anomalous behavior indicative of an IT issue.

    It’s now only a matter of time before providers of the platform widely employ machine learning algorithms to analyze machine data faster. There may even come a day when so-called “war room” meetings—that today need to be convened to identify the cause of an IT issue via a painstaking process of elimination—are no longer required.

    In the meantime, the percentage of applications that are instrumented in an IT environment needs to increase to achieve that goal. Fortunately, thanks to the rise of open source agent software, the cost of instrumenting applications is starting to decline. At the same time, DevOps teams should expect to see a marked increase in the volumes of incoming data, and they will need to find somewhere to store that data as well as determine the length of time it should be retained.

  • How Quantum Computing can Solve Real-World Problems

    How Quantum Computing can Solve Real-World Problems

    The history of technology can seem like it was very predictable in retrospect, but the future typically feels more uncertain. Nowhere is this uncertainty more evident than in the domain of quantum computing. When the spectrum of possible outcomes spans from “quantum computers will be one of the most important technology developments of all time” to “quantum computing may never really become practical enough to justify using over a classical computer alternative,” trying to make forecasts can seem futile. However, as a technologist, investor, consumer or just a curious observer, the boundaries and texture of the uncertainty itself can be very interesting, valuable and informative. 

    Quantum computing isn’t likely a replacement for today’s classical computing—at least not initially—but rather a potential complement to it for specific applications. A 2021 report by Deloitte lists dozens of applications that could be transformed by quantum computing approaches: Protein folding, fluid simulation, credit underwriting, financial risk analysis, supply chain optimization and forecasting, vehicle routing, fraud detection, fault analysis, weather forecasting, semiconductor chip design, product-portfolio optimization, consumer product recommendations—and this is only half the list. 

    Quantum computing could make previously intractable simulation, search and optimization calculations relatively easy and quick. For some kinds of calculations, a quantum computer could be mind-bendingly faster than a powerful classical computer could ever be. While today’s classical computers running machine learning systems can make incredibly accurate predictions, quantum-based systems can, in theory, far outstrip their speed at certain prediction-related tasks.

    The blend of physics, math and computer science behind quantum computers is fascinating, but complex. Computer scientist Scott Aaronson, one of the best science communicators, wrote a recent essay for Quanta magazine called, “Why Is Quantum Computing So Hard To Explain?” 

    To oversimplify, quantum computers are built around “qubits,” which are subatomic-scale components, often kept at incredibly cold temperatures (think: Absolute zero) to minimize interference from the rest of the universe. These components can be put into a quantum physics state called “superposition” that allows them to, in a sense, take on many potential values at once. Even though their value is uncertain, reliable calculations and transformations can be performed on these qubits while in a superposition before being reduced to a more classical state, effectively parallelizing computations. Qubits in a quantum computer can also be entangled with one another (a phenomenon that Einstein referred to as “spooky action at a distance”), harnessing them as a team to increase the variety of logical calculations that it’s possible to compute. For certain types of math—for example, searching for an optimal outcome from myriad options—quantum algorithms can seem close to instantaneous when compared to those that run on classical computing hardware. 

    Quantum Computing Challenges

    Publicly available quantum computing has been on the horizon for decades, yet there’s still no reliable forecast for when products capable of widespread adoption will get released, exactly what form the systems will take or how broad and deep quantum computing’s impact will be. There are many technical challenges when it comes to getting a broadly applicable quantum computer to be useful at scale. And quantum computers are racing against classical computers, which have also been improving very rapidly in power, versatility and ease-of-use. 

    One important bottleneck for quantum computers is that correctly reading the results of a quantum calculation is prone to a very high error rate—the delicate superposition state can deteriorate before the correct result is presented as an output.

    Aaronson explains it:

    The problem, in a word, is decoherence, which means unwanted interaction between a quantum computer and its environment — nearby electric fields, warm objects, and other things that can record information about the qubits … The only known solution to this problem is quantum error correction… But researchers are only now starting to make such error correction work in the real world. When you read about the latest experiment with 50 or 60 physical qubits, it’s important to understand that the qubits aren’t error-corrected. Until they are, we don’t expect to be able to scale beyond a few hundred qubits.

    Some theorists even believe a practical quantum computer is impossible to build because of the inherently high amount of “noise in the system.” Mathematician Gil Kalai predicts that doing error correction to reduce that noise to a usable level would, by basic computing theorems, restrict the computer’s number-crunching capacity so low as to be not worth using.

    Nevertheless, researchers continue to chisel away at the challenge of error correction and to pass key technical milestones. For example, in July, Google researchers published a paper in Nature explaining how they have found a promising error-reduction method that scales as more qubits are added to a system. 

    Hardware, Software and App Landscape

    There are several different approaches to quantum hardware. Rigetti and Oxford Quantum Circuits use superconducting quantum circuits, a more widespread approach favored also by Google, Microsoft, and IBM. IonQ (which is planning to go public via a SPAC transaction) and Honeywell (which recently announced a combination of its quantum computing unit with UK-based Cambridge Quantum Computing) use trapped ions. D-Wave uses quantum annealing, ColdQuanta is using cold atoms and Xanadu is using photonics. Each approach has its pros and cons, and each hardware technology faces a fluid set of shifting bottlenecks in reaching affordable scale. Work at the materials layer continues to advance as well, for example, Raicol Crystals is growing quantum quasi-phase matching crystals that can be used in applications as varied as quantum computing, sensing, encryption and communications. And it’s not certain that there won’t be another approach that leapfrogs them in solving the challenges of error correction and price.

    Other companies, such as Quantum Machines, Q-CTRL and Seeqc, are building control hardware that can operate in the space between classical and quantum hardware. Control systems can exist at temperatures warmer than absolute zero, but colder than room temperature. Some of these systems involve both software and hardware and can perform tasks like monitoring the quantum processor’s performance, performing error correction or serving as an abstraction layer connecting classical and quantum hardware. 

    There are also companies creating quantum computing application layer software, like Zapata Computing, Riverlane or QC Ware. This can include application frameworks, components, algorithms or even turnkey quantum computing applications. Classiq has software for quantum circuit synthesis. There’s also a lot of interesting fundamental research happening at the software intersection between quantum computing and artificial intelligence – e.g. Zapata published work recently using IonQ’s ion trap quantum system to train a machine learning system to generate high-quality handwritten digits with low error rates. 

    One of the first theoretical uses for quantum computing, proposed in 1994 by mathematician Peter Shor, would be to crack the RSA encryption widely used today to protect data. The threat spawned research into post-quantum cryptography, with researchers proposing multiple new approaches to encryption that would need to replace today’s algorithms to preserve the privacy of data. Companies such as ISARA (quantum-resistant encryption) and Quantum Xchange (an out-of-band, symmetric key delivery platform) emerged to help prepare governments and companies now for a world where quantum-based, encryption-cracking algorithms start to work on breaking today’s encryption.  

    There are several widely used software development kits (SDKs) that let developers write software for a quantum computer without actually having one to run it on. In fact, a kind of format war has already broken out between quantum computing development environments.

    One widely-used quantum SDK is IBM QISKit, an open-source Python-based development kit for prototyping software and devices on simulators or IBM Quantum Experience, a cloud-based quantum service. ExxonMobil, Goldman Sachs, Boeing, JP Morgan, Samsung and PayPal are just a few of IBM Quantum’s marquee customers. Google Cirq, another Python-based development kit, touts Volkswagen as a user, and Volkswagen is also a contributor to Google’s TensorFlow Quantum, an open source library that extends Google’s TensorFlow machine learning algorithms to quantum processors for rapid prototyping. Microsoft’s Quantum Development Kit (QDK) is based on Microsoft’s Q# programming language and brings quantum compatibility to Microsoft Azure cloud services. Early customers of Azure Quantum include Dow for chemical simulations and Toyota Tsusho, Toyota’s trading arm, developing traffic light optimizations to reduce urban traffic congestion.

    In a recent piece, The Information noted that “Amazon is looking to hire more than 100 new quantum computing scientists, hardware developers and engineers within its Amazon Web Services cloud business,” following the enormous amount of funding flowing to startups in the space, such as Quantum Motion, Atom Computing and PsiQuantum. Amazon also launched Amazon Braket, a fully managed quantum computing service that gives customers access to hardware from a diverse set of vendors such as IonQ, Rigetti and D-Wave. 

    It’s also possible that the most valuable quantum platforms won’t be versatile, horizontal platforms at all; rather, the space could evolve instead to include special-purpose machines, or quantum application-specific integrated circuits (ASICs). Unlike the way things have played out with classical computing, this could mean that quantum computers would be highly specialized integrated hardware and software machines tailored to specific applications, e.g. fluid simulation or fraud detection. 

    Early use cases may involve quantum systems taking specialized roles as part of a larger classical system, similar to how NVIDIA’s graphics processing units (GPUs), which often run alongside an advanced CPU, can be used to offload specific computations for which the GPU is well-suited. In this model, where quantum components are an “accelerator” to classical systems, a credit underwriter might use a quantum processor to handle one of the tasks involved in scoring applicants and might use a different special-purpose quantum processor to perform a piece of a calculation to optimize its borrower portfolio. 

    For many use cases, the error rates of calculations are still too high for developers to do more than hone their coding chops on quantum simulation platforms, waiting until quantum computers start to really work at scale. Quantum simulators, such as QMware, make quantum software algorithms and applications believe they are running on actual quantum hardware, but they are really running on classical computers pretending to be quantum computers. 

    In the meantime, to manage some of the timing uncertainty, quantum-related startups can pursue profitable projects with government, corporate and university buyers today as a source of funding and requirements. Like some biotech startups, they can do this while assembling a strong team and seeking an early adopter clientele, progressing through scientific milestones while keeping an eye on future commercial markets.  

    The hurdles to generally available quantum computers are much like those faced by alternative energy sources: Unlocking the potential will require breakthroughs in hardware design, lower manufacturing costs and a driving force to spur large-scale adoption. And many approaches won’t work. 

    And as with any technology, it is, of course, possible that software developers will find valuable applications for quantum processors that people haven’t thought of yet today—just as developers found that GPUs meant to handle graphics processing also excel at training highly parallelized machine learning models.

    That said, a broadly useful and effective quantum computer’s arrival has the potential to be as disruptive an opportunity as the creation of the Internet or the use of machine learning to make predictions.