Tag: audits

  • Ensure Your AI is Working for Your Data Team

    Ensure Your AI is Working for Your Data Team

    AI tools hold great promise for DataOps. They can help teams identify the best apps to migrate, identify optimum cloud infrastructure and even validate decisions. But how can an IT team ensure the AI tools it uses are actually advancing its DataOps goals?

    A hard pill to swallow is that even the best-designed algorithmic decision-making and AI tools are prone to errors. Fortunately, this is unlike errors in software development, which are largely caused by development. AI errors emerge from poorly defined project goals and the organization’s tendency to treat the technology as if it’s magic. Without solid requirements, model monitoring and regular check-ins with reality, it is almost impossible for AI to deliver.

    This is why audits must be a mandate when introducing any AI tool that greatly influences business decisions. However, before the audit is conducted, ensure that the following points are met:

    Ensure the AI Tool is ‘Auditable’

    This step ensures that organizations bring in the right AI tools to their arsenal.

    An AI tool is auditable when it is:

    • Transparent: When AI tools are used to enhance the decisions a business makes, stakeholders must be able to peer into the system to ensure the legitimacy of the algorithm decision. Ensure the AI tool you choose has a logging system that makes it transparent as to how the system is making a decision, what factors most influence it and how confident it is in that decision. This allows decision-makers to trust in the system, as well as creates a feedback loop for the developers to take in real-world data and enhance and fine-tune the system to your needs.
    • Built with best practices: Just as security engineers follow best practices when securing a network, AI models and tools built with best practices in mind fare better when in use. As a general rule of thumb, all AI models must keep detailed access and process logs, incorporate failback mechanisms and generate regular health reports.

    Set Up Markers for Success

    Don’t approach an AI audit as though it’s a sandbox question. If teams are clear as to what an AI tool must achieve and how to measure its success, only then will an audit be able to judge the performance of the tool.

    It is helpful to set up markers that describe success. This could be the number of cases the AI model has rightly predicted, the time it takes, the number of edge cases it is able to handle and the number of times the AI decision collided with a human decision.

    The Audit

    Once the AI tool has been integrated into the organization and is in active use for a defined period, audits must occur. They help you answer the following questions:

    • Is the algorithm suitably transparent to end-users?
    • Is it likely to be used in a socially acceptable way?
    • Might it produce undesirable psychological effects or inadvertently exploit natural human frailties?
    • Is the algorithm being used for a deceptive purpose?
    • Is there evidence of internal bias or incompetence in its design?
    • Is it adequately reporting how it arrives at its recommendations and indicating its level of confidence?

    (HBR: “Why we need to audit algorithms”)

    Assess the Algorithm’s Fitness for its Purpose

    From a use-case perspective, AI algorithms in modern software applications tend to fall into two main categories. One is offering insights that help humans make decisions—this is what AI-driven analytics is all about. The other is allowing software applications to undertake automated actions entirely on their own, as AIOps tools increasingly do.

    A basic step in auditing your algorithm is to determine which type of use case it is supposed to support, then evaluate how well it achieves it. This is important to do regularly, because AI use cases can drift as applications evolve and the original purpose of an algorithm might change over time. (It’s common for algorithms to be implemented originally as a way to offer insights, but for applications to be updated with the expectation that the algorithms can support automated decision-making.)

    Compare and Contrast Algorithmic Decisions Against Human Decisions

    Though some of the inner workings of an AI model are a black box to even data scientists, the result of those inner workings is not. An AI audit must set up a process and team that regularly and actively monitors the decisions AI makes and compares them against decisions that a human would make.

    This is useful in identifying early on whether the AI tool has internalized biases. With an AI system, input greatly affects learning. For example, if a tool used to forecast purchase orders is incorporated by the organization during peak sales season, the tool will assume that this is the organization’s baseline performance. This causes erratic predictions in the long run.

    Active Experimentation

    Though an audit should uncover any bugs, biases or security concerns of an AI tool, it also could be used to test system variables. An AI tool is generally built on assumptions of the problem borders and trends. However, assumptions are very fragile in reality. The baselines of reality could change, causing the AI to produce questionable results.

    An audit also must be able to revisit AI tools with new assumptions, annually or biannually, to uncover any new parameters that could help improve the accuracy of the tool.

    Interviewing Users of the System

    Ultimately, an AI tool is put into place to either reduce the workload of humans or enhance it. Daily users have vast knowledge and perspective into not just how the tool works but also its limitations and quirks.

    Conclusion

    More often that not, the mystique of an AI model is used as an excuse for erratic results and poor performance. By building a robust and transparent audit system, your organizations can ensure your AI tools are actually working for your teams—and if not, it can at least uncover rare scenarios and edge cases and help mitigate them.

    This sponsored article was written on behalf of Unravel.

    — Swaathi Kakarla

  • 8 Ways to Focus on Cloud Software Security

    8 Ways to Focus on Cloud Software Security

    Cloud Software Security Examined

    Cloud software security is critically important. Take, for example, health care: Doctors and other healthcare professionals are one of the biggest users of cloud software, yet recent studies demonstrate that more than 15 percent are considered to be operating at a high risk of security breaches, and more than half are operating at a medium risk.

    Security breaches can be devastating to almost any large industry. Companies must take accurate measures to ensure the safety of cloud computing software, as hackers are all too eager to break into a computer system and do what they seem to do best.

    Fortunately, there are a number of ways to increase security of data and apps in the cloud. Increasing cloud software security is almost like an insurance policy that affords valuable protection.

    Taking Steps to Increase Cloud Security

    The first step in building an effective cloud security plan is to conduct a complete internal audit that includes all current cloud providers. All internal security personnel should participate in the auditing process.

    Once the auditing process is complete, companies should take the following measures:

    1. Security personnel must take steps to ensure all company data is correctly encrypted. Having a secure encryption process in place will increase cloud security significantly.
    1. IT and/or security department should create an authentication process, which will help ensure all cloud software, as well as highly confidential internal information and related files, are secure.
    1. All new internal applications should undergo in-house developmental testing. Well-organized developmental testing procedures will guarantee that new applications are secure at the time they are released. This procedure is highly effective and also should increase cloud software security, when done correctly.
    1. Security solutions must be created that accurately scan and store various binary codes. What’s more, security solutions must be created internally. Runtime makes it extremely difficult to detect any security breach or potential vulnerability. Failure to detect or recognize a potential security flaw can prove disastrous for any organization.
    1. Use a web application scanning program for better security and safety. Use a provider that secures and monitors all web applications on a continual basis, as well as multiple scanning on a single platform. Multiple scanning will produce increased accuracy as well as unified results.
    1. Use one platform for all multiple assessment tasks to heighten cloud security. Using one platform will enable security personnel to identify and address an open-source risk quickly.
    1. Utilize streamline compliance centralization tools, which permit access to direct access to applications for compliance. Applications for compliance should include access to PCI controls and CWE. Both PCI and CWE can be used to make sure corporate compliance requirements are being met.
    1. Develop a system of continual compliance., which will monitor the system for any existing vulnerability or security concern. In addition, this unique system will fully access and examine any and all applications before they are released or deployed. Finally, the system of continual compliance will automatically conduct regular searches to identify all web applications that are related to a specific company domain. Any temporary sites also will be searched and properly identified. A continual compliance system is highly effective when properly utilized.

    There are many ways in which cloud software security can be enhanced. It may be a wise decision to consult with a security and compliance specialist. Many large and small organizations are now utilizing the services that security and compliance companies can offer.

    The Importance of Cloud Security Enhancement for Businesses

    In this of age of advanced computer technology it is becoming easier for hackers to access highly sensitive information from just about any organization. It is sad to say, but there are people who have made careers out of computer hacking. Many computer hackers are never brought to justice. Therefore, it is critical that organizations take the necessary steps to protect all computer software as well as sensitive company information that should never be released to the public.

    Having efficient software testing tools is the first step when looking to increase or improve cloud software security. Efficient testing tools will help an organization to better identify any security problem or vulnerability that may be present. If vulnerability is identified, the next step is to fix the problem and prevent additional security breaches in the future.

    Keeping company software and critical files safe must be a No. 1 priority. If critical internal information is compromised, it can cause severe damage that may be irreparable.

    About the Author / Reed Gusmus

    gusmusReed Gusmus the director of Marketing at QASymphony. Gusmus is an award-winning, passionate and innovative B2B marketing leader who excels at prescribing winning strategies for new customer acquisition and revenue-driving solutions for SaaS companies looking to get more from their marketing investment. Connect with him on LinkedIn and Twitter.

  • GRC: A Key Ingredient in DevOps’ Secret Sauce

    GRC: A Key Ingredient in DevOps’ Secret Sauce

    By now, surely everyone has seen cooking competition shows such as “Hell’s Kitchen” and “Top Chef,” in which people compete to create and serve the best dishes as quickly as possible. These hit TV programs are like the DevOps world, which also consists of people creating and releasing the best software as fast as they can with whatever they are given. In both instances, the best use of random ingredients brings rewards and failure brings disappointment.

    DevOps is becoming evermore common as companies in all industries work to unify their developers and other IT staff while automating as much of the process as possible, creating nimble, fast-moving teams. Think of it in the same way professional kitchens have cooks dedicated to specific stations, such as vegetables or fish, and are able to react swiftly to whatever orders are placed. By redesigning their in-house teams in this fashion, organizations can add speed, agility and resilience to critical functions.

    But, often what’s forgotten in the high-pressure, rush-rush-rush world is the critical importance of security and regulations. Governance, risk management and compliance (GRC) always must form the base decisions; otherwise; the final products will either not be able to be used or, worse, be dangerous to the company, its customers and its shareholders. It’s really no different than cooks having to abide by health code regulations or risk sickening and possibly even killing their diners. Don’t think GRC is that serious? It wasn’t long ago that Dwolla was fined $100,000 by the Consumer Fraud Protection Bureau (CFPB) for lying to customers about data risks. That was the CFPB’s first fine and you can bet more are sure to follow.

    Of course, a key difference is that in the business world, regulations and compliance standards constantly are being created or changed. That makes DevOps even more challenging, but that challenge cannot mean ignoring GRC. In fact, GRC should be seen as a key DevOps enabler, not an obstacle. An organization’s legal and compliance team can take advantage of highly agile, forward-moving DevOps teams to design and implement changes that satisfy risk and control points (resulting in successful audits and avoiding fines) while, at the same time, staying abreast or even beating the competition.

    The key here is to ground DevOps in GRC requirements so they become second nature and security measures are incorporated effectively into all DevOps processes, even automated ones. It’s crucial these measures are seen as integral to the processes and not simply layered on top as an afterthought, just as you wouldn’t try to get raw chicken juice off of lettuce after you’ve made a salad.

    Tactically speaking, how is this accomplished? DevOps teams need awareness and training. They need to learn the rules and laws specifically governing their particular industry, and how to implement them. And, companies need to have internal audits and other review policies in place to ensure GRC is enforced.

    For all of this to happen, however, a very important role must orchestrate it: management. Just as every kitchen depends upon a head chef to ensure not only that each station is obeying health codes (i.e. the law) but also that overall the meal is being prepared and delivered properly, company management must assume its authoritative role in setting up and defining the organization’s internal communication. This means breaking down silos and coordinating a regular flow of information from Compliance to DevOps team and back again.

    If this is done properly, DevOps can enable an innovational Nirvana, with companies boosting speed while maintaining security, creating the perfect consumer experience.

    About the Author / Robert Hawk

    Robert HawkRobert Hawk serves as Information Security Expert at xMatters, Inc. He has extensive experience in information systems security, computer security, cyber security, information assurance and governance, risk, and compliance (GRC) management. He specializes in frameworks and standards from: ISO/IEC, NIST, IEEE, IETF, ITU-T, Common Criteria, AMI-SEC, NERC, CIS, DoD, ANSI, PCI and ISECOM. Robert is a lifelong researcher, innovator and instructor. Connect with him on LinkedIn.