Attribute Inference Attacks

While Membership Inference Attacks (MIAs) focus on determining if a specific individual's record was part of the original training data, Attribute Inference Attacks (AIAs) address a different but equally significant privacy concern: Can an adversary deduce sensitive attributes about individuals represented in the original dataset by leveraging the synthetic data?

Even if an individual's exact record isn't replicated, and their membership cannot be confirmed via MIA, the synthetic data might still leak statistical patterns that allow an attacker to infer sensitive information (like income, health status, or political affiliation) given some non-sensitive attributes (like age, zip code, or occupation). The risk arises if the synthetic data makes this inference easier or more accurate than it would be otherwise (e.g., using only public information or the real data itself, if the attacker had access).

The Attribute Inference Threat Model

In a typical AIA scenario, the attacker possesses:

1. The synthetic dataset ($D_{syn}$): This is the primary tool for the attack.
2. Auxiliary information: The attacker knows some non-sensitive attributes for a target individual (or a group of individuals) whose records were potentially in the original dataset $D_{real}$. Let's denote these known attributes as $X_{known}$.
3. Target attribute: The attacker aims to predict a specific sensitive attribute, $Y_{sensitive}$, for the target individual(s).

The core question is: Does $D_{syn}$ enable the attacker to predict $Y_{sensitive}$ from $X_{known}$ with significantly higher accuracy than they could achieve without $D_{syn}$, or by using only $D_{real}$ (assuming they somehow gained access to a portion of it)?

Methodology: Training an Attacker Model

AIAs are typically implemented by training a machine learning model (the "attacker model") to predict the sensitive attribute. The process usually follows these steps:

1. Identify Target and Features: Define the sensitive attribute ($Y_{sensitive}$) to predict and the non-sensitive attributes ($X_{known}$) the attacker is assumed to possess.
2. Train Attacker Model on Synthetic Data: Train a suitable classification or regression model (e.g., logistic regression, decision tree, neural network) using $X_{known}$ as features and $Y_{sensitive}$ as the target variable, exclusively using the synthetic dataset $D_{syn}$. $$\text{AttackerModel} = \text{train}(X_{known\_syn}, Y_{sensitive\_syn}) \quad \text{where } (X_{known\_syn}, Y_{sensitive\_syn}) \in D_{syn}$$
3. Evaluate Attacker Model on Real Data: Test the trained AttackerModel on records from the original dataset $D_{real}$ (specifically, a held-out test set not used for generating $D_{syn}$). The performance (e.g., accuracy, AUC, F1-score) indicates how well attributes can be inferred for real individuals using patterns learned from the synthetic data. $$\text{Performance}_{syn \to real} = \text{evaluate}(\text{AttackerModel}, X_{known\_real}, Y_{sensitive\_real}) \quad \text{where } (X_{known\_real}, Y_{sensitive\_real}) \in D_{real\_test}$$
4. Establish Baselines: Compare $\text{Performance}_{syn \to real}$ against relevant baselines:
   • Baseline 1 (Real-to-Real): Train and test an identical attacker model solely on partitions of the real data. This represents the maximum inference possible if the attacker somehow accessed the original data. $$\text{AttackerModel}_{real} = \text{train}(X_{known\_real\_train}, Y_{sensitive\_real\_train})$$ $$\text{Performance}_{real \to real} = \text{evaluate}(\text{AttackerModel}_{real}, X_{known\_real\_test}, Y_{sensitive\_real\_test})$$
   • Baseline 2 (Prior Probability/Guessing): The performance achieved by simply guessing the most frequent class for $Y_{sensitive}$ or using other prior knowledge without leveraging correlations present in the data.

Quantifying Attribute Inference Risk

The privacy risk isn't measured by $\text{Performance}_{syn \to real}$ in isolation. High performance might simply mean the attribute is easily predictable in general. The risk attributable to the synthetic data is better understood by the difference or ratio between the attacker's performance using synthetic data and the baseline performance.

• High Risk: If $\text{Performance}_{syn \to real}$ is significantly higher than Baseline 2 and approaches $\text{Performance}_{real \to real}$, it implies the synthetic data effectively captures and leaks the sensitive correlations present in the original data.
• Low Risk: If $\text{Performance}_{syn \to real}$ is close to Baseline 2, the synthetic data doesn't provide much leverage for the attacker beyond trivial guessing.

The difference $\Delta_{\text{AIA}} = \text{Performance}_{syn \to real} - \text{Performance}_{\text{baseline}}$ quantifies the advantage gained by the attacker from the synthetic data. A larger positive $\Delta_{\text{AIA}}$ indicates a higher privacy risk.

Diagram illustrating the process of an Attribute Inference Attack using synthetic data to train a model, which is then evaluated on real data to assess risk.

Implementation Example (Pseudocode)

Let's consider a scenario where an attacker wants to predict a binary sensitive attribute has_condition (1 or 0) using age, zip_code, and occupation from a synthetic dataset.

import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score

# Assume D_real and D_syn are pandas DataFrames
# D_real: Original data
# D_syn: Synthetic data

# Attacker-known features and the sensitive target attribute
known_features = ['age', 'zip_code', 'occupation_encoded'] # Assume occupation is one-hot encoded
sensitive_attribute = 'has_condition'

# --- Attack using Synthetic Data ---
# 1. Prepare synthetic data for training
X_syn_train = D_syn[known_features]
y_syn_train = D_syn[sensitive_attribute]

# 2. Train the attacker model on synthetic data
attacker_model_syn = RandomForestClassifier(n_estimators=100, random_state=42)
attacker_model_syn.fit(X_syn_train, y_syn_train)

# 3. Prepare real data test set (ensure it wasn't used for synthesis)
# Assume D_real_test is a hold-out portion of D_real
X_real_test = D_real_test[known_features]
y_real_test = D_real_test[sensitive_attribute]

# 4. Evaluate the model trained on synthetic data against the real test data
y_pred_syn_to_real = attacker_model_syn.predict(X_real_test)
accuracy_syn_to_real = accuracy_score(y_real_test, y_pred_syn_to_real)

print(f"Attacker accuracy (trained on synthetic, tested on real): {accuracy_syn_to_real:.4f}")

# --- Baseline 1: Attack using Real Data ---
# (Requires splitting D_real into train/test if not already done)
# Assume D_real_train, D_real_test are available
X_real_train = D_real_train[known_features]
y_real_train = D_real_train[sensitive_attribute]

attacker_model_real = RandomForestClassifier(n_estimators=100, random_state=42)
attacker_model_real.fit(X_real_train, y_real_train)

y_pred_real_to_real = attacker_model_real.predict(X_real_test)
accuracy_real_to_real = accuracy_score(y_real_test, y_pred_real_to_real)

print(f"Baseline accuracy (trained on real, tested on real): {accuracy_real_to_real:.4f}")

# --- Baseline 2: Majority Class Guessing ---
majority_class = y_real_test.mode()[0]
accuracy_majority = accuracy_score(y_real_test, [majority_class] * len(y_real_test))

print(f"Baseline accuracy (majority class guess): {accuracy_majority:.4f}")

# --- Assess Risk ---
advantage_over_majority = accuracy_syn_to_real - accuracy_majority
leakage_ratio = (accuracy_syn_to_real - accuracy_majority) / (accuracy_real_to_real - accuracy_majority) if (accuracy_real_to_real - accuracy_majority) > 0 else 0

print(f"Attacker advantage over majority baseline: {advantage_over_majority:.4f}")
print(f"Attribute leakage ratio (relative to real data): {leakage_ratio:.4f}")

Interpreting the results:

• If accuracy_syn_to_real is significantly higher than accuracy_majority, the synthetic data provides useful information for inference.
• If leakage_ratio is close to 1.0, the synthetic data leaks almost as much information about the attribute correlations as the real data itself.
• If leakage_ratio is close to 0.0, the synthetic data offers little advantage beyond random guessing compared to what's learnable from the real data.

Attribute Inference Attacks provide a practical method for evaluating a specific type of privacy leakage. A high AIA risk suggests that the relationships between sensitive and non-sensitive attributes are too closely mirrored in the synthetic data, potentially compromising the privacy of individuals in the original dataset. This assessment is a critical component of a comprehensive privacy evaluation for any synthetic dataset intended for release or sharing.