Collaborative Evaluationedit
Evaluation of models and data processes using evidence from multiple parties.
Collaborative Evaluation refers to evaluation procedures in which multiple parties contribute evidence about model behavior, data quality, or distributional drift. In cross-silo settings, each participant has local observations but no participant has complete access to the global distribution.
Research contextedit
Evaluation across data silos is a networked problem, connecting Wasserstein geometry with AI and networks. A centralized benchmark assumes that all relevant data can be gathered and labeled in one place. Collaborative evaluation instead asks what can be inferred from partial, possibly biased local signals, especially when some parties operate in low-resource conditions.
Connection to Qiao's workedit
In When Sample Selection Bias Precipitates Model Collapse, collaborative evaluation is used to reason about recursive synthetic-data failure when the original data distribution is split across low-resource silos. The project uses distributional proxies, including Wasserstein-style geometry, to compare generated behavior against multi-party evidence. This connects Qiao's synthetic-data work to his broader systems interest: reliable AI often depends on how evidence is shared, not only on how a model is trained.