devopsy.kinsta.cloud

Tag: GDPR

  • Android Apps Must Let Users Delete Data ¦ RISC-V in the Data Center

    Android Apps Must Let Users Delete Data ¦ RISC-V in the Data Center

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: Google forces apps to make deleting users’ data easier, and the RISC-V drumbeat grows louder. (more…)

  • EU Data Privacy: Try Again | SkyPilot: Choose Cheapest Cloud | WFH: Better Meetings

    EU Data Privacy: Try Again | SkyPilot: Choose Cheapest Cloud | WFH: Better Meetings

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: EU-U.S. Data Privacy Framework reemerges, Berkeley brings cloud selection box, and proof that remote work is good work. (more…)

  • Data Centers IN SPAAACE | Discord GDPR Fine | AWS Fires Dead Wood

    Data Centers IN SPAAACE | Discord GDPR Fine | AWS Fires Dead Wood

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The EU wants to put servers in orbit, a GDPR penalty for Discord, and Amazon has the hatchet out. (more…)

  • App Store Antitrust Bill | GDPR vs. Google Fonts | Wordle Worth $1M+

    App Store Antitrust Bill | GDPR vs. Google Fonts | Wordle Worth $1M+

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: The Open App Markets Act polls well among devs, Germany fines a website for using Google Fonts, and the New York Times buys Wordle for an unfeasible sum.

    1. Antitrust Bill Would Open Up App Stores

    First up this week: The Open App Markets Act seems like a big hit with developers. In a survey, more then 80% wanted lawmakers to pass this bipartisan Senate bill.

    Analysis: But 30% of nothing is nothing

    One of devs’ biggest complaints is Apple’s 30% cut of all revenue—including in-app purchases. Many devs have also experienced random rejections of submissions to the stores. And these seem to be the main driving forces behind devs’ calls to tear down any walls around the gardens. But they should be careful what they wish for—users might not see it the same way.

    Karl Evers-Hillstrom: 8 in 10 app developers back measure to rein in Google, Apple

    The Coalition for App Fairness, [an] industry group for app developers, is pushing Congress to pass the Open App Markets Act. [The] bipartisan Senate bill … would block app stores from favoring their own in-house apps in searches, requiring developers to use their payment systems and preventing users from downloading apps from third-party stores.
    …
    Developers surveyed … complained about exorbitant fees … and expressed how they’d experienced difficulty getting their apps featured or accepted by app stores. Just 13 percent of app developers surveyed oppose the bill. … The margin of error is plus or minus 7.11 percentage points.
    …
    The proposal, co-sponsored by Sens. Amy Klobuchar (D-Minn.), Richard Blumenthal (D-Conn.) and Marsha Blackburn (R-Tenn.), is scheduled for a committee markup this week. The CEOs of Apple and Google have personally lobbied lawmakers to abandon [such] antitrust legislation.


    Wouldn’t that just kill the proverbial golden goose? No, says this Anonymous Coward:

    The only reason Apple has any success with the iPhone is because developers write apps for it, it’s no different to Windows in that regard. … If Apple’s app store is so good then everybody from developers to end users will continue to use it and nobody will use third party app stores.
    …
    Obviously there is a clear profit motive to exclude any competition in the app store space. But I don’t understand how people can claim that Apple makes great products and simultaneously argue that they couldn’t possibly survive competition on the platform.


    But Dr. Edith Müdyk—@BabyHueby—doesn’t sound hopeful this will go anywhere:

    Too bad policy isn’t dictated by public opinion.

    2. German Court Fines Site for Using Google Fonts Without Consent

    Did you know you need to get permission from a European user before you remote-load a Google Font? You do now.

    Analysis: Schrems II comes home to roost

    More GDPR popups ahoy! A German court has “clarified” that websites and apps should obtain informed consent before loading content—if loading it would reveal a user’s IP address to a third party. Luckily, the fine was only $110—this time, but if the perp was to do it again, the court threatened 2500× that amount.

    Thomas Claburn: Website fined by German court for leaking visitor’s IP address via Google Fonts

    A German court fined an unidentified website €100 ($110) … for violating EU privacy law by importing a Google-hosted web font. [It] passed the unidentified plaintiff’s IP address to Google without authorization and without a legitimate reason for doing so. And that violates … GDPR.
    …
    The website could have avoided this drama by self-hosting the font. … The ruling explicitly cites this possibility to assert that relying on Google-hosted Google Fonts is not defensible under the law.
    …
    The ruling directs the website to stop providing IP addresses to Google and threatens … a fine of €250,000 for each violation, or up to six months in prison, for continued improper use of Google Fonts. … These data privacy judgments … reflect the consequences of the EU Court of Justice’s 2020 decision to strike down the Privacy Shield data protection arrangements that had previously allowed US companies to exchange data with European partners under “Standard Contract Clauses.”


    Isn’t that ridiculous? And surely not the intention of GDPR? Aristos Mazer very much disagrees:

    This is very much the intended consequence, with a lot of analysis going into this regulation. … Even if that means redesigning foundations of the network.
    …
    Forcing sites to stop transferring data in any way shape or form to Google, Apple, Microsoft, Facebook, etc. … is exactly the goal. [It’s been] repeatedly validated by polls of EU citizens.


    And indeed, the point is to get “informed consent.” Here’s Hans Neeson-Bumpsadese:

    “To proceed beyond this point … your IP address needs to be shared with a 3rd party.” Would they click on that? … I know at least one person who would read that as, “If I click that link then you will tell people where I live.” (I know because they got a pop-up like that once, and phoned me in a state of mild agitation.)

    3. Wordle Sold to NYT for Over $1M

    Josh Wardle, a Welsh developer living in New York, has sold this winter’s social gaming hit to his state’s namesake newspaper of record. The New York Times thinks it’ll be a useful addition to its Games section. But the price tag is incredible.

    Analysis: Nice payday for Josh

    How much? Hard to see how the Times is going to make money out of this. Tomorrow’s word must be: NOROI.

    Alexis Benveniste: The Sudden Rise of Wordle

    In less than six months, Wordle has gone from a gift to a viral sensation, to … part of The New York Times. … Josh Wardle, a software engineer in Brooklyn, created the game as a gift for his partner … and it exploded in popularity.
    …
    Nearly two months later, 300,000 people played it. … The buzz around the game can be attributed to the spoiler-free scoring grid that allows players to share their Wordle wins across social media.
    …
    Wordle was acquired for an undisclosed price in the low seven figures.


    In other words, north of $1 million. Not bad for a quick, one-dev hack by Josh Wardle:

    The game has gotten bigger than I ever imagined (which I suppose isn’t that much of a feat, given I made the game for an audience of one). … I feel so grateful for the personal stories some of you have shared with me.
    …
    [But] I’d be lying if I said this hasn’t been a little overwhelming. [So] I am incredibly pleased to announce that I’ve reached an agreement with The New York Times. … I’m thrilled they will be stewards of the game.


    But “RIP Wordle,” mourns _xeno_, seemingly speaking for many:

    It’s about to be bogged down with trackers, and then later, ads. … A paywall is almost certainly on its way—The New York Times very carefully said it would “initially remain free.” … Not surprisingly, there are already plenty of clones. … There’s no way for the New York Times to make money off this.

    The Moral of the Story: No disguise will hide one’s true character.


    You have been reading The Long View by Richi Jennings. You can contact him at @RiCHi or tlv@richi.uk.

    Image: Masahiro Miyagi (via Unsplash; leveled and cropped)

  • A Developer’s Guide to CCPA, GDPR Compliance

    A Developer’s Guide to CCPA, GDPR Compliance

    Here’s what developers need to know to ensure compliance with the two biggest privacy laws

    The digital landscape is continuously evolving, and privacy regulations such as CCPA (California Consumer Privacy Act) and the European Union’s GDPR (General Data Protection Regulation) are in effect to give consumers their fundamental right to data privacy.

    These regulations force organizations to revamp their operations to comply. This means all departments within an organization, from marketing to software development and everything in between, have to keep privacy regulations in mind and tweak their workflows accordingly.

    In this article, we will discuss the steps developers can take to stay compliant with these regulations.

    Understanding Data Rights

    With more people concerned about their data rights, giving them complete control over their data is essential in today’s world. Under both GDPR and CCPA, here are all the rights consumers have concerning their data:

    • The right to be informed.
    • The right of access.
    • The right to rectification.
    • The right to erasure.
    • The right to restrict processing.
    • The right to data portability.
    • The right to object to processing.
    • The rights concerning automated decision-making and profiling.

    A consumer can practice these rights at any given time and enterprises are obligated to fulfill these requests as soon as possible.

    GDPR and CCPA recognize that more and more consumer data is available online, which increases the possibility of cyberthreats and invites other malicious activities. This is why it’s crucial for these regulations to protect the consumers’ data while dissuading any instances of data breach or sprawl.

    The Risks for Non-Compliant Businesses

    The European Union (EU) has a history of making an example out of companies that are non-compliant with its regulations. One of the EU’s most recent actions was against Google.

    France accused Google of infringement regarding the essential principles of the GDPR: transparency, information and consent. “Enforcement action was geared toward the way Google obtained consent,” said Myriah Jaworski, an attorney at Beckage PLLC.

    Google did not present how and why an individual’s data was collected and stored, nor did the company make it easily accessible. Google was fined $57 million by the EU.

    Seeing that even an industry giant is not immune to prosecution, it is clear that no company can get away with GDPR or CDPR non-compliance. To stay safe, developers must be well-versed in all the regulations and build their websites, apps and software with compliance in mind.

    CCPA vs. GDPR: What’s the Difference?

    While both laws serve to protect the rights of the individual, there are some differences between the two regulations. The following are the significant differences between the two laws.

    Who Needs to Comply

    The GDPR has a broad scope concerning who has to stay compliant with the law. It covers all citizens of the EU and regulates all organizations that collect and store personal information of EU citizens irrespective of their location and size.

    In contrast, the CCPA places constraints on the size of organizations that need to comply. It applies to organizations that have $25 million or more in annual revenue; possess the personal data of more than 50,000 “consumers, households, or devices”; or earn more than half of its yearly income selling consumers’ data.

    Financial Penalties

    The GDPR mandates penalties based on non-compliance and data breaches. These penalties can reach up to 4% of the company’s annual global revenues, or €20 million (whichever amount is higher), with the commitment that administrative levies will be applied proportionately. CCPA fines are not cumulative but instead are applied per violation, which can reach up to $2,500 per unintentional violation and $7,500 per intentional violation, with no upper cap.

    Consumer Rights

    Both regulations give the consumer specific rights that they can exercise. Some of these rights include the right to have information deleted or accessed. The GDPR specifically focuses on all the data related to European Union consumers, whereas the CCPA considers both consumers and households as identifiable entities. Businesses need to test their processes and ensure they can accommodate these rights.

    Use of Encryption

    The clauses on encryption in both laws constitute an area that, although similar, still have some differences. Both laws call for access to data encryption, making this an essential part of the privacy protection component for businesses.

    Steps to Compliance

    Developers are the front-line infantry in this struggle toward compliance because websites and mobile apps are the first interactions a consumer will have with an organization. It is essential to cover all bases from the start to make the compliance workflow as smooth and efficient as possible. Let’s take a look at the steps developers can take to comply with each regulation.

    Complying With CCPA

    Data Mapping

    To stay compliant, developers need to integrate proper data-mapping techniques into their systems. The law dictates that organizations should be fully aware of all the data they collect—this refers to what is collected, where it is stored and how it flows through the organization. Some operational suggestions include designating a single source of truth, maintaining lineage and tracking all data within the organization.

    Inform Consumers

    To comply with the CCPA, organizations will need the capability to fulfill data subject access requests (DSAR). A company’s website must show the consumer what data it is going to collect and how it will be collected. Developers can work with privacy officers to create a standard privacy notice for the website or an abbreviated pop-up policy at the point the data is collected.

    Verify Queries

    Organizations will be met with a flurry of requests from consumers exercising their rights under these regulations. Developers need to create a system by which the consumer can be authenticated and the correct information can be given to them. To streamline this process, developers can create a dedicated email account for requests and design workflows for verification purposes.

    Data Minimization and Purpose Limiting

    When collecting data, organizations need to make sure that the data is only used where necessary. To ensure that, developers can create forms that only require minimum information (data minimization) and organizations can make sure that internally used data is in line with privacy policies (purpose limitation).

    Data Security

    Under the CCPA, organizations are required to protect the data an organization keeps about a specific individual. Although not explicitly mentioned, it is beneficial for organizations to encrypt data at rest to prevent further compromise after any data breaches.

    Developers can ensure security by implementing robust applications that offer end-to-end encryption to protect consumers’ data.

    Complying with GDPR

    Efficiently Store Data

    The way an organization stores data can be the difference between compliance and non-compliance under GDPR. Developers need to ensure that minimal data is being derived from consumers to reduce liability and only store the data that is necessary for their processes. Lastly, developers should implement data subject access rights (DSAR) tools in their storage to efficiently respond to subject data access requests.

    Subject Access Requests

    Developers need to integrate a system that can map all the data in the data stores and make them easily accessible when consumers request access to the data that the company keeps, even complete deletion.

    Contacting Users

    Under the GDPR, an organization can not assume consent; it must be asked for. Developers working on a feature that will trigger an email or another message to be sent to users will need to integrate it with their organization’s consent tooling and check if they already have a consent channel for their use case. This will likely take the form of some source-of-truth database and an API that developers can query before sending messages.

    Profiling

    Profiling is the use of data to personalize a customer’s experience. To be compliant with GDPR, organizations should have a clear way for users to opt-out of profiling. The only important thing for developers going forward is understanding what counts as profiling and respecting a users’ choice before implementing any form of personalization.

    Rewrite your Privacy Policy

    The GDPR has brought several amendments to the current structure of any organization. The IT group is essential for organizations to revamp their privacy policies according to the GDPR. In this case, developers can integrate the privacy policy into company websites or as a pop-up to comply with the GDPR right to notice.

    Key Takeaway

    CCPA and GDPR are revolutionizing the data privacy sector, and organizations must comply with these regulations. Developers and marketers alike are going to have to find new ways in which they can efficiently comply with these regulations without hindering their current performance. Developers need to integrate automation to create a streamlined approach to compliance throughout the organization.

  • The EU’s Plans to Build a Wall Around GDPR-Protected Data

    The EU’s Plans to Build a Wall Around GDPR-Protected Data

    An ambitious German- and French-lead European Union (EU) initiative could wrestle tighter management, and eventually, control of data from commercial cloud providers.

    The project, dubbed as “Gaia-X,” is also intended to offer firewall-like protection for data sources from the EU. The initiative could have repercussions on how DevOps teams from outside of the EU manage data from European users, as well as from operations they might have that are physically based in EU member states.

    Without specifically citing U.S. cloud giants such as Amazon Web Services (AWS), Google Cloud Platform (GCP) or Microsoft Azure, German and French government officials said the initiative will allow organizations’ cloud data to benefit from EU sovereignty and security. Bruno Le Maire, France’s minister of the economy and finance (Ministre de l’Économie et des Finances), said during a televised Franco-German conference last week, for example, that Gaia-X would offer significantly more data protection for data stored on cloud services from EU organizations and users.

    Despite how General Data Protection Regulation (GDPR) mandates rules EU and international firms with user data from EU member state sources must follow—to avoid the risk of heavy fines and other penalties—the EU still lacks sovereignty in the application of GDPR, Le Maire said. Data stored on a U.S. cloud service provider’s server in the U.S. or overseas, for example, is still accessible by U.S. law enforcement officials in some cases under the Clarifying Lawful Overseas Use of Data (CLOUD) Act.

    Without disclosing technical details about the kinds of cloud services Gaia-X might offer, the initiative will adhere to four principles: “openness, interoperability, transparency and trust,” Le Maire said.

    “We are not China. We are not the United States—we are European countries with our own values,” Le Maire said. “We have our own economic interests that we want to defend.”

    EU data 1

    Over 20 organizations will form Gaia-X, including AtosBosch, Deutsche Telekom, SAP and Siemens. A prototype of the service should be available by end of year. While specific cloud services the organization will offer remains undisclosed, government officials announcing Gaia-X said it will offer artificial intelligence (AI)-aided meta analysis of data in such sectors as healthcare and smart city management. The use of AI services for data needs would also meet stringent privacy and ethical programming mandates, the officials said.

    In a statement, the German Federal Ministry for Economic Affairs and Energy (Bundesministeriums für Wirtschaft und Energie) said:

    “Predominant data silos, a lack of standards and the limited transparency of data infrastructure offers are today major obstacles to the application of artificial intelligence. To overcome these obstacles would be very costly and for this reason, the diverse know-how available in Europe is not pooled and used efficiently.”

    Build That Wall

    While created to ostensibly protect user data from the European member states and to enforce non-compliance with major penalties, accessibility by governmental authorities outside of the EU, for example, can remain outside of EU control. As mentioned above, for example, data storage that is ostensibly GDPR-compliant yet is on a physical AWS or another cloud provider’s server remains subject to access by U.S. law enforcement in certain circumstances.

    GDPR compliance is thus often considered a starting point for EU data sovereignty. In many ways, the EU’s initiative, pending the final details about Gaia-X’s infrastructure and the services it will offer, will serve as a firewall built to protect EU.

    Clive Longbottom, an analyst for Quocirca, agreed.

    “By creating a total data platform that is GDPR compliant, uses have the choice: build directly on that platform and not have to jump through hoops to prove compliance,” Longbottom said. “Those who choose not to, or due to geographical position, cannot use the platform directly can export data into Gaia-X and only have to prove compliance for that part of the platform outside of the environment. So, yes, EU data can be completely airlocked into Gaia-X – or can just make life a lot easier for users.”

    Instead of serving as an alternative to AWS, Gaia-X, based on the information the EU officials have communicated so far, is more of a data handling environment that may just have its own cloud components, Longbottom said.

    The idea, based on the available information “is to present an environment where data can be handled firmly within the needs of GDPR,” Longbottom said. “Now, there are fundamental difficulties around this, such as with data caching in the network, but we won’t let inconvenient realities get in the way of political perceptions.”

    The Gaia-X is thus “really meant to be a platform for data exchange that meets stringent requirements within the EU,” Longbottom said. “I would hazard that it will include those working outside of the EU that want to deal with countries, companies and entities within the EU, accepting data feeds from pretty much any cloud platform. It will probably be that once the data is accepted into the Gaia-X environment, it will then be deemed to be secure under GDPR—and that will not make it necessarily secure under GDPR end-to-end.”

    EU data 2

    Cloud Alternative

    A main selling point of the project is that once data adheres to Gaia-X compliance, DevOps could, in theory, benefit from more seamless data transfers between parties within the environment, as well as transparency about how and where it is stored.

    “Gaia-X is a good idea that will make life easier for EU members to share data across a trusted environment and allow externals to feed data into a known, highly standardized environment where they then do not have to prove GDPR compliance across Gaia-X itself,” Longbottom said.

    However, whether or not Gaia-X might one day compete against AWS, GCP or Azure remains to be seen. “If the EU can get more cloud platform vendors to join in and show how their platforms are GDPR compliant from the ground up, more EU companies will want to use such platforms where GDPR compliance is a given,” Longbottom said. “Meanwhile, most will stick with the big public platforms and use Gaia-X as a downstream GDPR data handling environment.”

  • Positioning Compliance as a Differentiator

    Positioning Compliance as a Differentiator

    As the threat landscape has become more perilous and complex, regulators have imposed a wide array of mandates designed to protect sensitive personal information. For most organizations, compliance is seen as the cost of doing business. However, if executed strategically it can not only improve a company’s overall security posture but shortens sales cycles and open the business to new markets.

    In order to turn compliance from a check-the-box line item into a valued business initiative, businesses need to identify all global, local and industry regulations that apply to their business and, also, strategically implement the processes and technologies that keep them compliant. Whether you’re targeting specific industry verticals or going after international customers, entering new markets requires continuous education about the latest in compliance and regulatory standards as they relate to data privacy and security.

    A good way to get started is to put together a roadmap for how you will get, and stay, compliant with the regulations relevant to your business. What follows is an outline of such roadmap.

    Start with the Basics

    When you’re building a house, a foundation is the key to a safe structure. This holds true for building a compliance roadmap for an IT infrastructure.

    The first step is to ensure the infrastructure is configured in accordance with Center for Internet Security (CIS) benchmarks. If operating in Amazon Web Services (AWS), ensure all AWS best practices are met. Setting this foundation will help meet basic security and compliance requirements and will simplify the compliance journey from the very start.

    Once the foundation has been set, it’s then time for the compliance, IT and security teams to determine which regulations apply to their business. This is the backbone of the compliance roadmap. The good news is that many of these regulations overlap so businesses can complete requirements for multiple regulations at the same time.

    First Stop: SOC 2

    As a component of the American Institute of CPAs Service Organization Control reporting platform, SOC 2’s goal is to ensure that internal IT systems are configured for maximum security, emphasizing the privacy of customer data. It’s specifically designed for service providers storing customer data in the cloud, and this means it’s becoming increasingly relevant as companies continue their cloud migrations. SOC 2 goes beyond a simple technical audit though, requiring businesses to establish and follow stringent security policies and procedures that encompass the security, availability, processing integrity and confidentiality of any data stored in the cloud.

    In order to achieve SOC 2 compliance, the organization first will need to establish an understanding of what normal looks like in their environment. It’s critical to baseline typical activity in order to be able to monitor for anomalies. This enables detailed audit trails that allow for deep, contextual insight into root cause, allowing organizations to remediate issues and thus keep up with SOC 2 requirements.

    The Roadmap Focal Point: GDPR

    The General Data Protection Regulation (GDPR) brought compliance into the mainstream. When GDPR passed, it established strict regulations for how organizations with European customers must handle customer data. The regulation is so broad, stringent and complicated that it has motivated many tech companies to create new job titles to ensure compliance.

    However, while there have been strict compliance regulations before, it’s the high financial stakes attached to GDPR that set it apart. A business can be fined up to 4% of its global revenue if it’s found to be non-compliant. Very few organizations can afford to take that kind of hit which is why so many make it the centerpiece to their compliance strategy.

    The privacy implications of GDPR are extensive but one of the most important and challenging requirements is the data breach notification. Organizations must notify authorities or specific data subjects within 72 hours of a breach. Most organizations are unable to locate sensitive consumer information within their environment, making this requirement near impossible. However, if the organization bakes data controls into its systems and enacts continuous monitoring and real-time intrusion detection, it not only becomes achievable but improves internal processes. 

    Go Broad with Certification Add-Ons

    Once you’ve done the basics, like SOC 2 and GDPR, organizations can then begin looking for compliance regulations that they can actually use as business differentiators. ISO/IEC 27001, for example, is a standard that SaaS organizations often use to demonstrate an aptitude for managing information security risks. The regulation formally specifies an Information Security Management System, a suite of activities concerning the management of information risks and lays out an overarching management framework to identify, analyze and address risks. Certification requires a host of documentation, including a clear information security policy, a risk assessment process and evidence of information security monitoring and measurement. The standard spans industry type, organization size and market, so it applies to just about any company.

    Compliance can be a powerful differentiator and business driver that inspires trust and confidence amongst prospects, customers and external partners. Although the above standards and regulations require extensive resources, non-compliance can result in fines and other punishment that can cripple a company. It’s important to remember that these compliance standards and regulations may have to be revisited, but once put into place and assigned to a dedicated compliance team, the once daunting task pays for itself.

    — Lindsey Hardy

  • Computer Vision is Building the Future

    Computer Vision is Building the Future

    Developing an accurate, regulated and purpose-driven facial recognition software is the challenge of computer vision engineering. It is also the future of how people do their everyday tasks: from unlocking a phone to signing into a bank app or logging the time they get to the office. Facial recognition amazes more and more with its new applications. Today, people can go to a concert without the tickets they bought online, they just have to bring…their face and show it at the gate.

    Computer vision science has been working on facial and object recognition software for some decades now. From the beginning, the main goal of this engineering, framed in the AI field, has been to give computers a visual understanding of the world.

    The goal gets fulfilled through a three-step process: image acquisition, image processing and image understanding. Each step can be done thanks to different kinds of technology. For example, image acquisition can be done using a web camera, an embedded phone camera or a 3D camera, among other types of lenses.

    With the help of algorithms computer vision is currently capable of recognizing and tracking objects. But it can go much further than that. Now, neural networks are at such a high level that they are capable of restoring and even creating new images. All of these features that seemed to be available only in futuristic or science fiction movies are now in the market thanks to AI algorithms, such as Generative Adversarial Networks (GAN).

    Today, big cloud providers have released their own AI and vision models including facial recognition APIs. Nevertheless, these models are conceived with rather general purpose algorithms that might be sensitive to specific environmental details, but don’t offer the possibility to retrain models with data gathered in a productive environment.

    The good news is some companies can offer AI models retraining in customers’ environment to accommodate any need or requirement, including usage of depth cameras or ToF sensors. But of course, these developments bring along some challenges. The way to succeed is to be ready and thoughtful in every step. This way, any surprises along the process will be controlled and won’t destabilize the project offered to a client.

    Last year at intive, we built a cognitive department with our internal AI experts as well as the best people we could find in the market. Eleven people shaped the core team that developed advanced facial recognition software in only one year of work. Our objective was to create an AI model fast suited to specific environmental conditions of small, medium-sized and big companies. We created AI models capable of using RGB, depth cameras and ToF sensors.

    Depth cameras enable the API to distinguish a real human face from flat images such as screens, tablets and paper. The new intive’s API can estimate age, detect sex and recognize a fraudulent face. It identifies eyes, nose and the ratio of length and distance of individual facial elements. It can even discern the differences between identical twins. The only face characteristics not taken into account are hair, mustache and beard; this is to make sure shaving or changing the hair style won’t impact the recognition capabilities.

    Here are the three major challenges for computer vision development.

    Legal and Compliance

    First, at any computer vision development, it is crucial to have the General Data Protection Regulation (GDPR) in mind. You must ensure you are allowed to manage or process certain biometric data. Imagine the city government installs cameras over the city parks, these are useful not only for recording but they are also capable of tracking people, thus enhancing the streets security. In this case, people’s identity has to be somehow recognized, and the way of doing this is by processing biometric data. If the person tracked does not know this is being done, a GDPR violation happens.

    Anytime a system needs to process people’s biometric data that can identify them, they need to give consent to process this data. To solve this, intive installed a button on our API that releases the biometric calculations and the person’s identification. By pressing this button, the person automatically gives consent to start the process. Additionally, while enrolling to the system, employees need to sign the consent.

    Same Environment, Always

    In every computer vision case, environment is an aspect you will always have to deal with. This refers to the technology used for acquiring images during the development, testing and productive stages of the project: the type of camera and its calibration, the drivers, libraries, settings and the light conditions.

    To ensure the API’s best accuracy, all stages must be developed with the same environmental conditions. Otherwise, the system loses its best accuracy and could come-up with worse results in the productive environment and in the development one. When environments change, the error rate rises.

    To ensure success, it is crucial to collect all possible data from the final environment and retrain the developed model with this data. For our computer vision project, we created several different production environments and we gathered all the data that came up. There will always be differences, for example, the lighting conditions are hard to control in every stage. But the more data you have, the fewer errors will appear.

    Set Acceptance Criteria and Meet It with the Data You Have

    From the beginning of the project, it is very important to set the accuracy you are aiming for. This means you need to decide how often the system recognizes an enrolled user correctly. To understand this better, if you have 10 users, and you set a classification accuracy of 90%, this means nine people are going to be recognized accurately.

    False positive rate (FPR) is linked to this and it needs to be decided too. This value represents how often the system recognizes a person that enters the system with different characteristics than the enrolled one. For example, if you are using the system to control employees’ entrance to an office, how often will a person who is not an employee (and not enrolled in the system) gain access to the building?

    Both values, accuracy and FPR, are strictly bound. Better accuracy is directly related to an increase in FPR. You need to decide if you want better accuracy and higher FPR or lower accuracy and lower FPR. This can be controlled smoothly along the course of the development if you set it according to the available data.

    Accuracy and FPR will change based on the quality of data–the photos used. The more pictures (data) per enrolled person, the more accurate the system is. Pictures from different angles allow the system to have a better chance of being effective. In our case, we took a minimum of five pictures for every enrolled person.

    Working on a computer vision project with facial recognition software can seem intimidating. But it is just a matter of being well organized. Take into account the three challenges highlighted above and you will be on the right track to success.

    — Grzegorz Tymiński

  • Data Mapping and GDPR: How Are They Related?

    Data Mapping and GDPR: How Are They Related?

    Every business enterprise receives data from an array of diverse data points. The volume of data is growing, and businesses understand the importance of leveraging data and converting them into actionable insights. Owing to data diversity, it becomes important to integrate them in a standard format. This is where data mapping takes a precedence. It helps establish relationships between different data models. Although data mapping offers several business benefits, it is also an integral aspect of the General Data Protection Regulation (GDPR) compliance. This post discusses the concept of data mapping and how it helps you adhere to GDPR.

    What is Data Mapping?

    As the name suggests, data mapping is a term for mapping the journey of data in an organization. A data map answers questions ranging from the origin of data to its journey through the organization. Today, most business enterprises are using various types of data mapping software, including ETL data mapping tools, to achieve their data mapping targets. Depending on the capabilities of these software applications, data mapping is used to perform a wide range of data management tasks such as data integration, data migration, data warehousing and data transformation.

    How Does a Typical Data Map Look Like?

    Typically, a data map includes the following information:

    • Type of data collected
    • Basis of data processing (personal or sensitive)
    • Purpose of data collection
    • Storage period of data
    • Place of storage
    • Conditions of data storage
    • Various data transfer destinations
    • Location of third party vendors (especially, international third party vendors)
    • Protocols followed during the data transfer (to international third party vendors)
    • Legal basis for data processing

    GDPR Compliance and Data Mapping

    The GDPR mandates business enterprises to ensure fair handling and security of the information they own. To assure the security, it is important to track the data journey from its point of origin to a conclusion. Without a detailed insight into data lifecycle, it is difficult to implement any security. Thus, data mapping becomes an essential step to achieve GDPR compliance. The following GDPR articles and their requirements will help you understand the importance of data mapping.

    • Article 30: Maintaining a written record of data processing activities in an organization is mandatory. A data map serves as a best written record, which organizations can easily retrieve and present at the time of GDPR audit.
    • Article 25: Data mapping helps organizations showcase their commitment towards data privacy.
    • Article 6: This article governs personal data processing under a lawful basis. Data mapping helps organizations to show the basis of processing–personal or sensitive.
    • Article 28: Data maps enable GDPR auditors to understand the extent of third party access to organizational data. All organizations are required to maintain a list of third-parties with whom they might be sharing their business data for various reasons.
    • Article 35: This article discusses Data Protection Impact Assessment (DPIA), and data maps enable business enterprises to meet this requirement effectively. To analyze the risk of a particular process, an organization needs to understand the data flow across an organization–how the data is being collected, where it is stored, who uses the information, how long it will be retained and so on.

    How an Efficient Data Mapping Software Helps in Achieving GDPR Compliance

    Now, what is an efficient data mapping software? It can be any application that aids a business organization to exploit the benefits of data mapping while adhering to the GDPR regulations. The following are a few important features of such GDPR compliant data mapping tools, which helps distinguish them from others.

    • Ability to Identify and Map Personal Data: Advanced data mapping software applications, which may or may not involve ETL data mapping software, allows users to identify and map personal data flow in an organization. They are designed to handle a large influx of data. These solutions utilize metadata to identify various data elements. The metadata helps in creating data maps that answer questions ranging from the origin of data, type of data collected, data subjects involved in process, service level agreements, among others. Thus, advanced data mapping software applications enable users to maintain, manage and monitor data maps for business efficiency and GDPR compliance.
    • Helps Ensure Data Security: Professional-grade data mapping software solutions are equipped with various security features that enable users to secure their database while providing them access to data, whenever required. They also allow organizations to conduct risk analysis of their data and ensure adherence to DPIA.
    • Assures a Better Control: Earlier reference to GDPR articles suggest how GDPR mandates enterprises to ensure data protection while handling personal data. An efficient data mapping software will help them address this need. They enable users to modify their applications to establish control, as well as safeguard personal data effectively.

    Achieving GDPR compliance is not an easy task. However, you can easily address it at various stages and it will help your business adhere to privacy standards while effectively meeting customer demands. You can take advantage of various data mapping software applications including advanced ETL data mapping tools to create effective data maps, stay compliant and, above all, drive your business forward.

    — Chandra Shekhar