UDLCO CRH: What is the scientific validity of synthetic data to understand patient centred research questions around their illness interventions and outcomes?

 

Summary

Introduction

• Background & Problem: Synthetic Health Data (SHD) is increasingly promoted as a privacy-preserving mechanism to share clinical datasets and foster innovation under stringent data governance frameworks (e.g., India's Digital Personal Data Protection Act, 2023). However, a core epistemological tension exists between basic privacy preservation, population-level statistical smoothing, and the individual-level temporal realism required for clinical research.

• Objective: Analyze the conversational dialogue between hu1 and hu2 to evaluate whether synthetic data generation algorithms preserve individual patient trajectories, examine how SHD differs from de-identified case reports, and assess the scientific and legal implications of adversarial risks like Membership Inference Attacks (MIAs).

Methods

• Qualitative Conceptual Analysis: A Socratic dialogue between interlocutors (hu1 and hu2) scrutinizing definitions of synthetic data, comparing HIPAA de-identification against generative modeling, evaluating manual versus algorithmic abstraction of Electronic Health Records (EHR), and assessing privacy threat models.

Results

• Delineation of Synthetic Data: Unlike HIPAA de-identified case reports—which strip direct identifiers from actual records—synthetic data generates entirely new records that mimic underlying mathematical distributions without a direct 1:1 match to real individuals.

• The Precision Medicine Gap: Current population-level generative algorithms prioritize global distributions (e.g., aggregate age, mean lab values). When longitudinal patient trajectories are not preserved, the synthetic data loses clinical coherence (e.g., temporal order of interventions vs. outcomes), rendering it unsuitable for precision medicine questions.

• Adversarial & Regulatory Exposure: Generative models that attempt high-fidelity reconstruction risk overfitting. This creates vulnerabilities to Membership Inference Attacks (MIAs), where malicious actors confirm whether a target patient's data was used in model training. Under statutes such as the DPDP Act 2023, data fiduciaries face severe financial liabilities (up to ₹250 crore) despite acting without malicious intent.

Discussion

• To be scientifically valid for patient-centered research, synthetic data must preserve individual trajectory fidelity—capturing temporal dependencies, causal sequences, and multi-variable dynamics over time. However, doing so pushes generative models toward memorization, directly elevating privacy leakage risks. Achieving scientific validity for precision medicine while ensuring legal compliance remains an unresolved trade-off in synthetic health data architecture.

2. Key Words

• Synthetic Health Data (SHD)

• Patient Trajectory Fidelity

• Precision Medicine

• Longitudinal Clinical Realism

• Membership Inference Attacks (MIA)

• Data Fiduciary Liability (DPDP Act 2023)

• Causal Outcome Modeling

3. Socratic Steelman Thematic Analysis

Thesis Statement

Preserving individual patient trajectory fidelity in synthetic health data is a scientific necessity for precision medicine and causal research, yet it creates a privacy and legal vulnerability by increasing susceptibility to overfitting and Membership Inference Attacks.

Theme 1: The Epistemic Imperative

Why Preserving Individual Trajectory Fidelity IS Necessary for Scientific Validity

The Steelman Argument

Disease progression and therapeutic response do not occur as isolated static snapshots; they unfold sequentially within an individual body over time. For synthetic data to answer patient-centered research questions (e.g., "How does treatment X alter the 5-year progression of condition Y in patients with biomarker Z?"), the data generator must preserve temporal sequence, dose-response timing, and cross-variable correlations within single trajectories.

If an algorithm flattens data into population-level averages, it produces "clinically impossible" patients—such as an individual receiving second-line chemotherapy before receiving a primary diagnosis. Without individual trajectory fidelity, synthetic data may achieve statistical harmony at the population level while remaining scientifically invalid for precision medicine.

Socratic Inquiry

• Socrates: "If a synthetic dataset correctly matches the population mean for glycemic levels and the overall percentage of prescribed metformin, but assigns those prescriptions randomly across time points without regard to individual patient blood sugar spikes, is the dataset scientifically valid?"

• Interlocutor: "No. While the macro-level distribution matches, the internal clinical logic is broken."

• Socrates: "Then can we conclude that population-level statistical similarity is insufficient for clinical research?"

• Interlocutor: "Yes. Scientific validity in healthcare depends on temporal causality at the individual level."

Theme 2: The Privacy & Liability Paradox

Why Preserving Individual Trajectory Fidelity IS Problematic & Risk-Prone

The Steelman Argument

High-dimensional individual trajectories are uniquely detailed. When a generative model is trained to replicate these complex, multi-year clinical journeys with high fidelity, it must capture nuanced combinations of features. The closer a synthetic trajectory mirrors real-world complexity, the closer the generative model comes to memorizing its training data.

This memorization opens the door to Membership Inference Attacks (MIAs). An adversary possessing partial data about a real patient can probe the synthetic dataset or model to verify whether that patient was part of the original cohort. Under strict regulatory regimes like the DPDP Act 2023, this leak reclassifies the synthetic dataset as personal data, exposing data fiduciaries to massive legal liabilities. Thus, high trajectory fidelity directly threatens the primary purpose of using synthetic data: safe data sharing.

Socratic Inquiry

• Socrates: "If a synthetic patient trajectory must be unique and complex enough to mirror a rare patient's disease journey, how does it differ in practice from a model that has overfitted and memorized that real patient?"

• Interlocutor: "Mathematically, the boundary becomes blurred. High fidelity requires capturing unique feature combinations, which inherently increases disclosure risk."

• Socrates: "Then does maximizing scientific utility for precision medicine inherently compromise privacy guarantees?"

• Interlocutor: "Precisely. Perfect trajectory fidelity and absolute privacy against inference attacks are fundamentally in tension."

Structural Comparison: Scientific vs. Privacy Trade-Offs

Trajectory Dimension	High Trajectory Fidelity	Low Trajectory Fidelity
Primary Research Utility	Precision medicine, causal modeling, trajectory analysis	Macro-epidemiology, health economics, system testing
Clinical Coherence	Preserves temporal logic and treatment-outcome sequences	Risks clinically impossible co-occurrences and smoothed dynamics
Privacy & MIA Risk	High vulnerability to memorization and inference attacks	Low vulnerability; privacy protected via distribution smoothing
Legal/Regulatory Exposure	Elevated liability under DPDP 2023 if membership leaks	Reduced regulatory exposure due to low re-identification risk

Provide an imrad summary, key words and a socratic steelman thematic analysis of the content below focusing on why and why not preserving fidelity of the individual patient trajectory is necessary for scientific validity of synthetic data to enable better understanding of patient centred research questions around illness interventions and outcomes

Conversational transcripts:

[22/07, 09:47]hu1: In this article of my 'The Future of Healthcare Data' series, I explore how Synthetic Health Data can help balance innovation with responsible data governance.

The article discusses:

• What Synthetic Health Data is and how it differs from real patient data

• Why it is becoming critical for Artificial Intelligence, clinical research, and software testing

• How synthetic data enables collaboration while reducing privacy risks

• Why synthetic data is not automatically risk-free and still requires strong governance

• The importance of high-quality healthcare data in generating accurate synthetic datasets

• How Synthetic Health Data supports privacy-by-design and aligns with the DPDP Act, 2023 and the DPDP Rules, 2025

Read full article: https://lnkd.in/dxVC7SRu

[22/07, 15:47]hu2: This is very useful

If we look at what this review means by synthetic data, we may realise that real patient data shared through scientific case reports that follow a HIPAA guide-lined de-identification process are actually already following quite a bit of those strategies that render a real patient data synthetic?

For example:

the review defines synthetic data as a set of information around patients containing their:

Age distributions.

Laboratory value patterns.

Medication usage.

Clinical outcomes.

Healthcare utilization

The above mentioned data are all taken from real patients even in synthetic data but elements that remotely can make the patient identifiable are removed?

My question here is: what more do we need to remove from scientific case reports to make it fit the current above definition of synthetic data?

[22/07, 15:57]hu1: That's a very insightful observation.

The key distinction is that HIPAA de-identified case reports are still derived from real patients, whereas synthetic data are newly generated to preserve statistical and clinical patterns without representing any actual individual. 

So it's not about removing more information but ensuring that no record is traceable to or reconstructs a real patient while retaining clinical realism.

[22/07, 19:15]hu2: While we understand the current need for synthetic data is essentially to protect real patient privacy, what are the real elements in that synthetic data that still provide it some kind of scientific credibility and what would make it scientifically untenable?

I have gathered a few answers to the above from other team members and colleagues working in this area and I'm looking forward to your and others inputs here

[22/07, 20:34]hu1: Scientific credibility comes from preserving the underlying statistical distributions, clinical relationships, and outcome patterns while ensuring no record corresponds to a real patient. 

It becomes scientifically untenable when those relationships are distorted, clinically implausible, or fail to represent the intended population.

[23/07, 07:52]hu2: Thanks.

We need more information on how one may be able to do that in individual patient data sets.

Currently is the process of generating synthetic data hidden behind a black box AI algorithm? Perhaps because it's doing it as part of larger population based datasets where the individual patient record doesn't gain prominence in which case of our insights remain restricted to population based average outcomes that cannot answer precision medicine questions related to the individual patient's clinical relationships and outcome patterns?

Manually if one were to understand this process let's say being done by a human one patient record at a time, then it's likely that the human would take the real patient data from the EHR and begin by stripping it of all identifiers and yet preserve it's clinical relationships, outcome patterns and statistical distribution when the cases start adding up?

The most important step in the above manual exercise would be to preserve fidelity of the individual patient trajectory, without which, that patient's reality would be distorted and consequently cause total distortion of clinical relationships and outcome patterns and statistical distributions?

Do our current synthetic data generating algorithms ensure fidelity to real patient events data trajectories?

[22/07, 20:39]hu1: Membership Inference Attacks are one of the most important privacy risks to consider when evaluating the quality and safety of synthetic data.

Synthetic data is intended to preserve the statistical properties of the original data without reproducing actual individuals.

However, if the synthetic data generator overfits the training data, it may produce records that are extremely similar to real patients. An attacker can then infer that certain individuals were included in the source dataset.

https://www.linkedin.com/pulse/understanding-membership-inference-attacks-healthcare-sujeet-katiyar-rnqsf/

[23/07, 08:12]hu2: Thanks! This was again very useful and considerably complicates the above discussion on synthetic data and its scientific validity, efficacy, benefits vis a vis its risks and harms.

👏👏

The bottom-line appears to be that scientific endeavours are often powerless against malicious endeavours and as you've aptly well illustrated, malice can result from the data principal's (patient's) intent to hide incriminating information from insurance companies and the insurance companies even more malicious strategies to verify the patient's claims by attacking other inferential pathways that can see through locked health vaults with their presumed safety!

Innocent bystanders such as data fiduciaries (innocent only in terms of bearing no malice for either of the previous two parties) can get penalized upto Rs 250 crores just for being a fiduciary for the sole purpose of trying to answer their research questions around patient illness events and intervention outcomes!