Signals: Mapping Keys

Introduction

This document is crafted to guide new signal providers through the process of initiating signal transmission to Intuizi and facilitating their successful onboarding. At Intuizi, we prioritize data privacy and security; accordingly, we do not store Mobile Advertising IDs (MAIDs) or Internet Protocol Addresses (IPs) on our cloud servers. Instead, our operational model relies on utilizing the cloud storage of our signal providers. Through this setup, we can process requests to “decrypt” Encrypted Mobile Advertising IDs (EIDs) or Encrypted IPs (EIPs), maintaining the integrity and confidentiality of user data.

Key Mapping

For effective data synchronization between Intuizi and the signal provider, it is essential to establish a mapping system. This system should correlate the Encrypted Identifier (EID) with the Mobile Advertising ID (MAID) or Encrypted IP (EIP) with The Internet Protocol Addresses (IPs), necessitating the storage of mapping keys on the signal provider’s cloud server. These keys must consist of two fields: the MAID and the EID and/or IP and EIP.

Definitions

• MAID (Mobile Advertising ID): A unique identifier allocated to mobile devices for advertising purposes, enabling the tracking and recognition of users to deliver tailored advertising experiences.
• IP Address (IP): A unique numerical identifier assigned to devices connected to a network, allowing for device recognition and data exchange. IPs are used to enable analysis of online connectivity and device associations. 
• EID / EIP (Encrypted Identifier): An encrypted form of the MAID or IP, secured via salting and the MD5 hashing algorithm to enhance data protection while still facilitating personalized advertising while keeping results distinct from each provider.

Encryption Process

The conversion of MAID/IP to EID/EIP is executed using the MD5 hashing algorithm, a well-established cryptographic hash function that generates a 128-bit (16-byte) hash value, predominantly utilized for verifying the integrity of data. The resultant EID/EIP is to be presented in uppercase, comprising 32 characters.  For privacy and security, we treat Mobile Advertising Identifiers (MAIDs) and IP addresses equally, applying the same standards and practices. Unique salts are applied to both MAIDs and IPs, transforming each into distinct keys. These keys are securely stored and can only be re-identified within a 90-day period. After this time, keys are removed automatically, meaning any identifier not re-encountered within 90 days becomes completely anonymous.

Encryption Code Samples

To assist in the encryption of MAID to EID across various platforms, the following code samples are provided:

• AWS Athena: UPPER(TO_HEX(MD5(TO_UTF8(mobile_ad_id))))
• Google BigQuery: UPPER(TO_HEX(MD5(CAST(mobile_ad_id AS STRING)))) 
• MySQL:  UPPER(MD5(mobile_ad_id)) 
• PostgreSQL: UPPER(MD5(mobile_ad_id::text)) 
• SQL Server: UPPER(CONVERT(VARCHAR(32), HASHBYTES(‘MD5’, mobile_ad_id), 2)) 
• Oracle: UPPER(RAWTOHEX(STANDARD_HASH(mobile_ad_id, ‘MD5’)))

Data Organization

Partitioning

To enhance query performance and streamline the overall process, mapping files should be partitioned by date, following the day=YYYY-MM-DD format.

File Format Recommendations

We advocate for the utilization of Parquet files, structured to include two fields: EID/EIP and MAID/IP, both designated as string types. While CSV and GZIP formats are also acceptable (provided they are comma-separated), additional support is extended to AVRO and ORC formats.

Sample Data Entries

123e4567-e89b-12d3-a456-123456789000, F21260D66361B24E2A5EBD68E9C162AA
123e4567-e89b-12d3-a456-123456789001, C5FF54F2A94FD1B7D122E2E22789FBA7
123e4567-e89b-12d3-a456-123456789002, 9C74810B65846CF9404D96C7030395CC
123e4567-e89b-12d3-a456-123456789003, 5703F3030833E7502CA800614C1969A0
123e4567-e89b-12d3-a456-123456789004, A46BDA17217882EAC2F08AB00A9852B8

Cloud Storage Configuration and Access

In alignment with our introductory note, Intuizi does not host MAIDs on our cloud servers. Signal providers are thus required to establish their storage solutions, choosing between AWS S3 Storage and Google Cloud Storage as their data repository.

AWS S3 Integration

Signal providers must provide the following details:

• Bucket Name: A distinct identifier for their designated S3 bucket within AWS.
• Region: The geographical location of the S3 bucket, pivotal for optimizing data transfer and adhering to compliance mandates.

Granting Permissions to Intuizi

To enable Intuizi to export data to the client’s AWS S3 bucket, specific permissions must be accorded to our user ARN. This is achievable by incorporating our ARN into the bucket policy or IAM policy, detailed further in the accompanying JSON example.

Google Cloud Storage Integration

For those opting for Google Cloud Platform (GCP) storage solutions, the following details are requisite:

• Service Account Key: A JSON key file corresponding to a service account with requisite Google Cloud Storage access permissions, essential for authentication and authorization purposes.
• Bucket Name: The designated name of the Google Cloud Storage bucket for data uploads, facilitating precise identification within the GCP.
• Activation of Google Cloud Storage API: It is imperative for clients to ensure the activation of the Google Cloud Storage API within their GCP project, enabling programmatic interactions with the storage infrastructure.