Category: IT as Code

  • Building Trust Into Your Product by Design

    Building Trust Into Your Product by Design

    In today’s world, every tool, every solution and every technology has the potential to do both good and bad. Good, if used according to the way it was designed, but bad if it falls into the wrong hands, or is manipulated or abused. As the ‘Peter Parker’ principle says, “With great power comes great responsibility.”

    As the COVID-19 pandemic continues and consumers look to digital communications to maintain some sense of normalcy and connectedness during these desolate times, the onus is on us, industry leaders, to ensure that the products and services being developed to connect and communicate with disparate employees, customers, and partners aren’t weaponized by bad actors.

    The best way to do this? By building trust into your design.

    You Have to Define Trust to Know How to Build Trust

    According to Merriam-Webster, the dictionary definition of trust is “one in which confidence is placed.” But the reality is that in today’s world, there are many ways trust can be perceived.

    Think of the automotive industry, where trust can be tangibly seen in the growth and success of certain car brands over decades. There, trust can be seen as ‘brand loyalty,’ or customers who repeatedly purchase from the same brand because they trust the company and trust the product.

    Trust in technology is the same. It has to be engineered and cultivated, but at the end of the day trust in the product must be mirrored by consumer trust in the organization. The two go hand in hand.

    The best way to approach building and sustaining trust when developing new products boils down to three key things: integrity, third-party accountability and consistency. Integrity stems from delivering high grade products and solutions, in conjunction with demonstrating accountable, transparent communications at the organizational level. Third-party validation further underscores that integrity, and restates the quality of products and services being delivered from a third-party perspective. And finally, consistency; being able to deliver exemplary products and solutions on a continual, consistent basis – the most essential ingredient in maintaining and bolstering trust among customers.

    Transparent Communication is Key

    As we enter 2021, the threat landscape is fraught with bad actors looking to take advantage of the latest threat vectors and emerging technology. As our lives continue to transition from physical to digital, and as more personal information is stored online, companies have to prioritize proper and timely communication about cybersecurity incidents.

    This brings us back to organizational trust being foundational for trust in a product. If you’ve seen the Netflix documentary ‘The Social Dilemma’ you know that social media is an oft-cited example of technology being misused or manipulated by bad actors. But after Twitter suffered a social engineering attack in July, the platform issued a swift update to its users. They were open, prompt and transparent in their analysis of the event, and highlighted not only the updated safety precautions put in place in response to the breach, but also underscored their ongoing commitment to trust and transparency within the Twitter community.

    Incidents like these, though harmful in the short term, can be less detrimental if addressed and communicated in a timely, transparent manner. There’s a saying, ‘It’s not what happens to you, but how you react to it that matters.’ The same goes for proper breach disclosure within the tech industry as a whole.

    Trust has Never Been More Foundational to the Business of Building

    As we enter 2021 and the COVID-19 pandemic rages on, more companies than ever are finding themselves in the business of building. Healthcare providers are drumming up new ways to digitally engage with patients and providers outside the physical confines of a hospital. Financial services organizations are adding more immediate, personalized ways to engage with customers when, where and how they need it. Educators are creating new digital solutions to keep millions of children, learning from home for the foreseeable future, engaged. And retailers are rebuilding the in-store experience for customers entirely online.

    With so much building going on around us, it’s important to keep in mind that the building process today requires not only outlining what you’re trying to build and defining how you intend to use it, but also analyzing how it can potentially be abused, manipulated or rendered ineffective.

    Consistently running your products and solutions against the latest threat models is one way to ensure your products’ security posture is up to date with the latest attack vectors. Another idea is to incorporate ‘security champions’ into your product development process and teams, to act as mini consultants who evaluate potential cybersecurity vulnerabilities. Once you’ve incorporated methods like these, turning to a third party to assess and sign off on your product’s cybersecurity resiliency is a good way to validate your products’ security adherence and effectiveness in the face of ever-evolving threats.

    Trust and Development Should go Hand in Hand

    At its core, trust and development should go hand in hand. Developers don’t need to recreate the wheel every time – there are plenty of libraries available online to leverage and pull from. What’s most important is for developers to understand the libraries they’re using to build, and ensuring that proper security protocols are built in from the start.

    Trust can seem like a huge lift, or an unnecessary distraction on the way to an MVP, product launch or update. I assure you, this is not the case. No matter how big the effort may appear, investing in trust – and the various methods, ways and changes that harden your products and services against abuse – will pay dividends down the line. Customers will trust your product and your company, and will, therefore, be more likely to remain loyal. If nothing else, keep in mind that building trust takes time, effort and consistency, while losing trust can happen in the blink of an eye.

  • ZeroNorth Adds Analytics App to Advance DevSecOps

    ZeroNorth Adds Analytics App to Advance DevSecOps

    ZeroNorth today added an Advanced AppSec Risk Analytics module to its platform for orchestrating DevSecOps processes.

    John Worrall, ZeroNorth CEO, said the analytics tool pulls vulnerability data generated by a variety of security tools to provide both DevOps and cybersecurity professionals with a single source of truth for assessing security within an application environment.

    The ZeroNorth platform is designed to enable IT organizations to orchestrate multiple vulnerability scanning tools within the context of a larger DevSecOps workflow, and can apply analytics to a large pool of security data. That data can be viewed in the context of specific groups, such as a business unit or specific application development and deployment team, or within the context of whichever application generates the most revenue for the business, Worral said.

    Additionally, the Advanced AppSec Risk Analytics module will surface types of vulnerabilities detected and the vulnerabilities most encountered, ranked by criticality, along with other trends in types and number of vulnerabilities detected/remediated, such as the number and criticality of vulnerability findings per specific scanner. It also tracks the applications and entities scanned, number of vulnerabilities detected per application/entity and the applications/entities that have the highest risk.

    ZeroNorth

    Armed with that analytics data, Worral said it becomes easier for IT teams to prioritize and focus their efforts, in addition to identifying areas of the business that may need additional cybersecurity resources and training. IT leaders can also get a snapshot view of the top application security risks to better assess the organization’s overall security posture, Worral noted.

    The Advanced AppSec Risk Analytics module will also surface vulnerability remediation bottlenecks spanning multiple applications within the context of a software development life cycle (SDLC), Worral added. Longer term, Worral said the data collected by the ZeroNorth analytics application will also create an opportunity for the company to apply machine learning algorithms to DevSecOps.

    In general, Worral said the only viable path toward achieving DevSecOps is to foster more collaboration between developers and cybersecurity teams. Getting those teams on the same page in terms of the security data that needs to be analyzed is a crucial first step, Worral said.

    It’s still early on in the adoption of DevSecOps best practices within organizations, however, it’s apparent a lack of cohesion between software developers and cybersecurity teams about which vulnerabilities to prioritize is a major source of dysfunction. Cybersecurity teams tend to create long lists of vulnerabilities discovered within applications without providing much context. Developers are then left to weigh, on their own, when to address a vulnerability versus focusing on writing additional code needed to make an application delivery deadline. Developers take potential vulnerabilities seriously, but there are only so many hours in a day to write code.

    Unfairly or not, developers are being held more accountable for application security. The challenge – and the opportunity – is to find a way to make the entire DevSecOps workflow a whole lot less contentious by helping DevOps and DevSecOps teams more aware of which tasks should be prioritized when.

  • Rewind Acquires BackHub to Protect GitHub Repositories

    Rewind Acquires BackHub to Protect GitHub Repositories

    Rewind, a provider of a software-as-a-service (SaaS) platform for backing up cloud services, today announced it has acquired BackHub to gain control of a platform for backing up GitHub repositories.

    Based in Germany, BackHub adds a data protection platform to a Rewind portfolio that includes offerings for backing up everything from QuickBooks to Shopify. Fresh from raising an additional $15 million in funding, Rewind CEO Mike Potter said BackHub addresses a critical issue involving the protection of some of the most valuable intellectual property any organization has in a cloud service. GitHub, a subsidiary of Microsoft, is the largest host of source code in the world, with over 40 million users and 190 million repositories. BackHub claims it protects data in those repositories for more than 1,000 organizations.

    Rewind GitHub backup

    While GitHub itself is a reliable cloud service, the code residing in GitHub is subject to the same potential threats as it would be in any other cloud service. It’s not uncommon for a malicious actor to gain access to credentials that they exploit, for one reason or another, to delete or alter data – or, in the case of GitHub, code. BackHub makes it possible to back up code stored in GitHub to another cloud service where it can be easily recovered.

    Potter said the ultimate goal for Rewind is to make it possible to back up the entire cloud. Achieving that goal will require both additional acquisitions and internal product development, said Potter.

    In the meantime, the company is focused on providing data protection for specialized cloud services usually overlooked by other backup and recovery tool providers. For now, it leaves the backup and recovery of data stored on widely used cloud services, such as Microsoft 365, to other vendors. Rewind claims more than 80,000 organizations are using its platform to protect data, stored in a wide range of cloud services, that address a specific niche use case.

    Given the prominent role GitHub plays in software supply chains, it is not surprising that the code stored in Git repositories becomes a target for cybercriminals. Much of the intellectual property stored in those repositories is a high-value target for organizations looking to reduce the amount of time required to re-engineer a rival company’s functionality.

    Nation-states around the world have cyber-espionage programs aimed at gathering any type of digital asset that might benefit their military or economy. Naturally, copying code isn’t always enough when, in one fell swoop, add code that might derail that project long enough for them to catch up.

    Unfortunately, there have been some high-profile breaches involving software supply chains. The best way to determine if a project has been compromised is to compare code bases. Repositories, by definition, provide application development teams with version control capabilities. However, backing up a GitHub repository provides one extra layer of security that can make it easier to recover from a breach.

  • EraDB Launches Log Management Alternative

    EraDB Launches Log Management Alternative

    EraDB today launched EraSearch to provide IT teams with an alternative to open source ElasticSearch software for managing logs at scale.

    Todd Persen, EraDB’s CEO, said EraSearch is based on a modern database that decouples storage and compute. In this way, EraSearch can scale more efficiently than ElasticSearch, while providing IT teams with the same familiar construct for managing logs.

    Persen said EraDB also uses true zero-schema data representation, and an indexing engine based on machine learning algorithms, to enable EraSearch to scale better. This approach also cuts cloud and operational costs by as much as 30% and 70%, respectively, by relying, in part, on object-based storage systems to store log data.

    As IT environments become more complex and continue to scale, organizations that standardized on ElasticSearch are discovering that a platform based on decades-old search engine technology is not keeping pace with their needs.

    EraSearch provides an alternative, based on a commercial time-series database platform that EraDB created, to address such use cases as an alternative to database platforms such as InfluxDB.

    The question is whether IT organizations, that have relied on ElasticSearch for years, are ready to abandon an open source platform in favor of a proprietary platform, even if it provides a similar user experience. EraDB, however, is betting there are enough IT organizations willing to make that shift to justify its investment in EraSearch.

    Persen said one issue with ElasticSearch is that, as log volumes increase beyond a few terabytes, the platform becomes inefficient and difficult to efficiently scale. That leads to higher costs in terms of both raw storage, as well as all the manual effort on the part of software engineers, required to maintain the platform.

    EraDB, of course, is making a classic case for a proprietary platform over an open source platform: that open source, while free, often proves more expensive to support. Persen said EraDB decided to eschew the open source model altogether, because it is too difficult to prevent nullification of the benefits open source could bring to a small company by open source project forks.

    The ElasticSearch community, meanwhile, is roiled by Elastic’s decision to change the licensing terms under which it makes ElasticSearch available. Elastic is shifting away from an Apache license to a dual-license strategy that includes a Server-Side Public License, or SSPL, along with an Elastic License. That shift was made to help ensure more of the revenue generated by Elastic search winds up in the hands of Elastic, rather than cloud service providers, such as Amazon Web Services (AWS), that routinely launch paid services based on open source code. AWS responded to the Elastic move by announcing plans to fork ElasticSearch code that it will now curate.

    Regardless of how that battle ends, however, EraDB is making it clear it views its platforms as too critical to its customers’ IT operations to get caught up in a similar maelstrom over business models and associated licensing terms.

  • Dynatrace Extends Reach of Observability Platform

    Dynatrace Extends Reach of Observability Platform

    Dynatrace this week updated its observability and software intelligence platform as part of an effort to expand its reach across modern enterprise IT environments.

    Announced at the online Dynatrace Perform 2021 conference, the company’s namesake platform can now capture logs from Kubernetes and Red Hat OpenShift as well as Amazon Web Services, Google Cloud Platform and Microsoft Azure cloud platforms.

    In addition, Dynatrace is adding support for open source log data frameworks such as Fluentd and Logstash. Those logs can now be centrally viewed via a Dynatrace Log Viewer that enables IT teams to search, analyze and segment both real-time and historical log data.

    Dynatrace also added a Cloud Automation Module based on Keptn, an open source tool for automating configuration of observability tools, orchestrating code tests and quality checks. Keptn was created by Dynatrace and is being advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    The company also launched a Software Intelligence Hub to make it easier to access more than 500 pre-configured integrations, as well as access tools for creating integrations without having to write code.

    Dynatrace also announced it has extended a previously launched Application Security Module to now include support for Node.js applications. In addition, the module includes the ability to enforce security policies against applications instrumented with its agent software, and offers extended capabilities to enable the detection of Kubernetes cluster vulnerabilities, as well.

    Finally, Dynatrace has added a Session Replay capability to its Digital Experience Module to enable IT teams to capture a mobile application experience.

    Steve Tack, senior vice president of product management at Dynatrace, said the company is moving quickly to add support for platforms and technologies widely employed within cloud-native applications to provide IT teams with a single platform through which they can manage monolithic and microservices-based applications.

    With the rise of microservices, IT environments are becoming too complex to manage without some type of observability platform. Historically, cost concerns limited IT teams to employing application performance management (APM) platforms, such as Dynatrace, only for their most mission-critical, monolithic applications. It’s now become apparent that observability platforms capable of correlating log and telemetry data captured from applications and infrastructure are required to effectively manage a modern IT environment.

    As part of that effort, Dynatrace is now collecting data from additional sources alongside the agent software it provides. That data is then fed into an observability platform, that Dynatrace has augmented with a Davis AI engine, to identify anomalies and other potential issues by employing machine learning algorithms.

    It’s not clear to what degree IT teams completely trust the recommendations surfaced by any AI engine. However, as platforms such as Dynatrace collect more data from an IT environment, the more accurate AI models tend to become. In fact,  IT teams that don’t have access to AI capabilities might soon be at a distinct disadvantage, compared to other IT teams that do.

    In the meantime, the battle for observability supremacy continues to intensify, as incumbent providers of APM platforms such as Dynatrace look to forestall a host of startup rivals, all promising greater visibility into IT environments at a lower cost. The challenge now is determining what level of visibility is achieved, and at what total cost.

  • New Relic Adds Visualization Tool to Observability Platform

    New Relic Adds Visualization Tool to Observability Platform

    New Relic announced today it has added a visualization tool, dubbed New Relic Explorer, to make it simpler for IT professionals to discover the root cause of issues that adversely affect application performance.

    Buddy Brewer, global vice president and field CTO for the Americas, New Relic, said the goal is to make it simpler for IT professionals to navigate and, ultimately, act on, all the metrics collected by the New Relic One observability platform.

    New Relic Explorer includes a New Relic Lookout capability that enables IT teams to drill down into telemetry data gathered from New Relic One, as well as other third-party sources, and a New Relic Navigator capability for determining the health of relationships between services, said Brewer. These capabilities make it possible to more easily identify correlations, abnormal history and traces that would be, for example, indicative of blind spots or unknown relationships between IT services, Brewer said.

    Just as importantly, IT professionals can more easily discover recent changes to an IT environment, which are most likely the cause of any new issue that is adversely affecting the IT environment, Brewer added.

    New Relic Explorer requires no configuration, and will be included as part of the existing New Relic Full-Stack Observability offering to give IT teams a single unified view of their IT environments, said Brewer.

    As IT environments become more complex, the amount of time spent looking for the root cause of an issue that might only take a few minutes to fix has substantially increased. As a result, organizations are allocating a significant amount of time to convene “war rooms” to identify the source of problems; those problems can now be more readily identified using New Relic Explorer.

    In general, Brewer said, New Relic remains committed to opening up the platform to external telemetry sources, including data collected using open source tools, such as Prometheus and OpenTelemetry agent software, both advanced under the auspices of the Cloud Native Computing Foundation (CNCF).

    New Relic, like providers of similar tools, has recently begun repositioning itself as a provider of an observability platform, rather than of application performance management (APM) tools, that provides insights into both applications and the infrastructure on which they depend. That approach promises to provide IT teams with more context any time an issue arises. As part of that shift, New Relic is making it clear that while it views its agent source as a richer source of telemetry data, its platform can consume relevant data wherever it can be found. New Relic will also remind customers that its agent software, and the other data collection tools it provides, are simpler to install and update.

    The tradeoff, of course, is cost. While open source software itself is free, the total cost of implementing and managing that open source software can be substantial. Regardless of approach, however, the days when IT teams only applied application monitoring tools to their most mission-critical applications because of cost concerns are coming to an end. It has become impossible to manage a modern IT environment made up of both monolithic and microservices-based applications without such capabilities.

  • AIOps Builds Organizational Efficiency, Survey Finds

    AIOps Builds Organizational Efficiency, Survey Finds

    Enterprises entered 2020 with big plans. Post-pandemic, AIOps could help them achieve those goals, according to a new research survey from Digitate and CorporateLeader.

    Enterprises entered 2020 with the best-laid plans; many had efforts lined up to expand their use of cloud services and continue their steady efforts to digitally transform their businesses. Then the pandemic happened, and business and IT teams found themselves scattered and with a new playbook: make the organization as effective as it can be while working remotely.

    This sudden shift to remote work exacerbated existing challenges within organizations, from managing hybrid environments, personnel management challenges and teams finding themselves struggling to adjust. Meanwhile, enterprises still had to focus on staying ahead of their competitors technologically. All of these challenges meant there was no room for wasted or time-consuming efforts to manage the underlying business-technology systems.

    To help increase efficiencies, some contend that the answer is artificial intelligence for IT operations, or AIOps. Proponents of AIOps say that AI is needed to augment human decision-making today because of the complexity of modern enterprise environments. Today’s environments often find their operational data spread across on-premises and multiple cloud systems. The result is difficulty in managing infrastructure issues as they arise, and the considerable time it takes to identify and fix IT issues.

    According to a survey of leading European IT analysts and advisories, conducted in fall 2020 by operations automation software provider Digitate and executive network forum CorporateLeaders, the drudgery of managing routine tasks was the most commonly cited reason for turning to AIOps, which was followed by “no proactive capability of detection and correction of system issues” at 64% and “no flexibility to scale with business needs,” also cited by 64% of respondents.

    When respondents were asked to identify the key benefits of AIOps, they highlighted common strategic and operational issues that hinder performance. According to respondents, these include the removal of manual processes (91%), improvements in agility and reliability (82%) and reach predictability (73%). More swift problem resolution also was cited by 73% of respondents.

    In a case study shared within the CorporateLeaders report, Marion Fokkinga, technology advancement manager at Netherlands-based semiconductor maker NXP explained that NXP’s move to AIOps was driven by the company’s swift expansion. “Since 2015, we have been busy with multiple merger and acquisition projects,” Fokkinga explained. “As a result of this, there was less time for innovation and transformation in IT and in the business.”

    The company set out to automate routine operations tasks with AIOps so that their resources could be freed to focus on higher-value tasks. According to Fokkinga, AIOps enables tasks, such as report generation, that previously consumed half a workday to be completed in several minutes. Saving time is the most significant result, said Fokkinga. With AIOps, NXP reports a 97% improvement in incident response time and a 95% reduction of manual effort in some of its supply chain planning work.

    Of course, in the current treacherous business environment, there’s a lot more at stake than simply building enterprise operational efficiencies. Over time, it may come down to the very success of the organization. According to the survey, a full 82% of respondents said that AIOps is necessary to the future growth and transformation of their business. “Respondents who went on to give qualitative answers later in the survey explained that the complexity of hybrid and multi-cloud infrastructures, increasing transactional volumes and the criticality of online business systems make it impossible for traditional IT systems to manage every application and infrastructure without recourse to automation. It was highlighted that the overwhelming amount of data in today’s Operations are simply beyond human analysis capacity,” the report stated.

    One respondent said that, through AI, the performance of some tasks and decisions were improved, as routine tasks were eliminated. That has to be welcome news to enterprises struggling to remain competitive while simultaneously striving to remain productive and agile during a global pandemic.

  • Understanding Data Storage: Lakes vs. Warehouses

    Understanding Data Storage: Lakes vs. Warehouses

    Now more than ever, companies are looking for new ways to incorporate data analytics into their daily operations and leverage data-driven insights to improve business functions. They are frequently turning to complex data for tasks like machine learning and artificial intelligence, which are becoming necessary to understand and reach customer segments across industries. However, understanding data storage is a key factor in developing a successful strategy. The two most common storage formats are data lakes and data warehouses – but there are benefits and pitfalls to each that organizations must understand in order to properly capitalize on them.

    Data Lakes vs. Data Warehouses Use Cases

    One effective way to better understand the different functions of data lakes and data warehouses is to focus on the end use case. Typically, there are three types of data consumers: farmers, explorers and executives.

    • Farmers need data and information to execute their day-to-day activities. They must report on the key performance indicators (KPIs) required to execute their job by providing data in a structured format.
    • Explorers are users who want to experiment with data, look at new types of data pools and gather specific insights. If data is presented in a fixed format, it hinders their progress, as they seek data that doesn’t have a pre-defined structure.
    • Executives are responsible for making business decisions, and often require information presented at different aggregates, with the ability to narrow in on data subsets as needed. However, they require information that is somewhat structured, and speed is critical for them.

    If we look at these perspectives more broadly, they essentially boil down to two key types of data usage – the exact needs that data warehouses and data lakes are designed for. Farmers and executives are largely served through data warehouses, which require information to be timely and structured. Some of the main use cases for data warehouses include operational analytics, business intelligence and predictive analytics for data science and machine learning applications. Explorers are best served through data lakes, since the data formats are not predetermined. Data lakes are often most effective for use cases like a centralized data catalogue that enables organizations to view all sources of data in a single place, or recent advances in AI application development that require the processing of unstructured data, such as text, images, video and audio.

    Challenges and Convergence of Storage Formats

    Despite the differences, most companies can use both data lakes and data warehouses for their various analytics needs. However, to successfully capitalize on the benefits, it is also important to understand the challenges. In particular, data lakes present difficulties when it comes to parsing data. They were initially designed to capture data across the enterprise in its natural format, without enforcing schema, so that users could garner more insights; a fundamental aspect in creating a data-driven culture across the organization. However, without a predetermined use case, data lakes can quickly become data swamps. End users were historically unable to figure out whether the data was stale, and ownership was not well established. This has been remedied by taking a use case-driven approach and only including data that has defined use cases or ownership.

    Now that many of the issues surrounding data lakes have been resolved, the two storage formats must complement each other to meet the varying needs of the customer. This has contributed to blurred lines between data lakes and data warehouses. Data lakes are now capable of schema enforcement and answering rapid business intelligence queries, which were traditionally qualities of data warehouses. Data warehouses have separated compute and storage and can read directly from big data file systems, enabling users to read semi-structured data. As such, we are rapidly moving toward integrated data environments and the convergence of data lakes and data warehouses.

    Supporting Data-Driven Initiatives

    For initiatives like artificial intelligence and machine learning to succeed, data must be presented as an immutable entity that can be used for experimentation. With data lakes, it is crucial to separate the data into different zones and maintain a refined zone after transformation. Then, in the refined zone, companies can enforce schema and allow the schema to evolve to ensure the data is ready for machine learning and data science needs. Most importantly, they must catalog the data and enforce metadata management, data quality and governance. Data warehouses, on the other hand, excel in providing data sets ready for discovery and consumption. Companies should integrate these data sets with an interactive data catalog so they are discoverable – this is the most important step in making artificial intelligence and machine learning possible.

    If organizations can learn how to best capitalize on data warehouses and data lakes for their intended purposes, they will be well equipped to uncover key data-driven insights to help guide their business strategy. This will help enable use of other advanced technologies and help transforming them into data-driven companies.

  • CNCF Graduates Open Policy Agent Project to Manage Compliance as Code

    CNCF Graduates Open Policy Agent Project to Manage Compliance as Code

    The Cloud Native Computing Foundation (CNCF) announced this week that the Open Policy Agent (OPA) project, which many IT teams are employing to manage compliance as code, has officially graduated.

    Torin Sandall, co-founder of the OPA project and vice president of open source at Styra, whose compliance management platform is based on OPA, said formal recognition of OPA alongside other CNCF projects, such as Kubernetes, should help further adoption of the open source project that first took shape in 2016.

    Now being advanced under the auspices of the CNCF, the open source agent created by the OPA project is increasingly being incorporated into a wider range of compliance and security platforms. At the same time, more developers are starting to manage compliance as code across a wider range of applications. The ultimate goal is to make it easier to meet compliance requirements using a declarative framework so that responsibility for implementing compliance policies shifts further left toward developers.

    OPA uses a general purpose engine for enforcing policies that uses a set of rules that developers embed within applications using the Rego language. OPA has gained traction among cloud native application developers because it provides a means to enforce policies across a wide range of microservices.

    Sandall said going forward, the primary focus of the OPA project should be to develop and share as many integrations as possible. There are many instances where a use case for OPA has been created, but development teams simply don’t know that use case exists. As a result, many IT teams are attempting to employ compliance as code to address an issue that others have already solved, Sandall said.

    During its time as an incubation project managed by the CNCF, the security special interest group (SIG) within the CNCF conducted two external OPA security audits. The OPA project has also defined a security vulnerability disclosure process and a security response team.

    It’s not clear whether chief compliance officers within enterprise IT organizations have developed an appreciation for OPA. In theory, many compliance issues could be avoided if organizations mandated using OPA – or another approach to managing compliance as code – within their applications. Those compliance issues, of course, are multiplying as the number of compliance mandates expand globally. The challenge is that most compliance teams don’t have much visibility into the application development process.

    In the meantime, as DevSecOps processes continue to mature, it seems almost inevitable they would expand to include compliance mandates, most of which only exist to ensure some base level of security is present. Of course, developers may not want to be responsible for compliance mandates. However, the sooner compliance issues are addressed in the development process, the less likely they are to become DevOps’ problem when deployed in a production environment.

  • Lightstep Delivers Next Generation Observability Platform

    Lightstep Delivers Next Generation Observability Platform

    Lightstep this week announced general availability of a Change Intelligence observability platform based on a time-series database that is capable of processing a trillion events each day.

    Ben Sigelman, Lightstep’s CEO, said as IT environments become more complex, thanks to the rise of microservices and cloud-native computing platforms, existing monitoring and observability tools will not be able to keep pace with the rate of change in what have become highly dynamic IT environments.

    While AIOps promises to address that complexity by employing machine learning algorithms to analyze IT environments and surface recommendations, Sigelman said there will always be the need for observability platforms that enable IT teams to discover the root cause of an issue in a way that surfaces the insights needed to fix the problem in real-time.

    The Lightstep database was built by the same team that created Monarch, the globally-distributed in-memory time series database system that Google employs to monitor its applications and systems. Lightstep’s database makes it possible to collect metrics, in the form of distributed traces, at a level of scale that allows IT teams to analyze events across a distributed IT environment made up of both monolithic and microservices-based applications, said Sigelman.

    Lightstep is the latest entrant in an already crowded field of observability platforms, all seeking to consolidate the wide range of monitoring tools IT teams have used for decades.

    The challenge IT teams face is that each of those monitoring tools is focused on a narrow range of platforms. IT teams often lack context when an issue arises, and wind up trying to correlate reports from multiple monitoring tools in the hopes of discovering the root cause of an issue. Observability platform providers are making the case for a single platform that correlates application and infrastructure events in a way that makes it easier for IT to fix issues faster.

    Observability, of course, has been a core tenet of DevOps for years. The tools for achieving that goal, however, have been cumbersome to implement and maintain using legacy application performance management (APM) platforms. Most of those platform providers, have rolled out what are positioned as next-generation observability platforms. Now, IT organizations will need to decide if they want to maintain investments in existing platforms versus moving to one of the raft of upstarts looking to dethrone APM incumbents.

    The startups are counting on a massive expansion in demand for observability platforms. IT organizations have tended to limit their usage of APM platforms to mission-critical applications due to cost and the challenges associated with maintaining the agent software needed to collect metrics from each application. Modern observability platforms have automated much of the metrics collection process, while open source software promises to make metrics collection more affordable..

    The challenge, of course, is that many organizations are already struggling to manage complex IT environments, and can’t necessarily wait for that open source software to mature.