This is a local, offline preview rendered against this benchmark's own data, using the
real open-source OpenEBench visualizer component -- it is not connected to
openebench.bsc.es and does not reflect a live, published Benchmarking Event. Page
organization follows the real OpenEBench benchmarking-event page structure
(community → benchmarking event → challenges → leaderboard), referenced from
the public CAID benchmarking page.
Community
ELIXIR 3D-BioInfo Community benchmarking activity on scoring and ranking of protein-protein interfaces and quaternary structure assignment, based on a curated benchmark of 1677 homodimer crystal structures.
protein-protein interactionquaternary structureinterface scoringhomodimersstructural bioinformaticsAlphaFold-multimer
Benchmarking Event
Challenges
PPI-PhysioClassification — Discrimination of physiological vs. non-physiological (crystallographic) homodimer interfaces
PPI-PhysioClassification challenge leaderboard
AUC-ROC vs. Accuracy-at-threshold · 14 participants
Community: 3D-BioInfo
Challenge: PPI-PhysioClassification
Metric X: AUC-ROC
Metric Y: Accuracy@threshold
Full metrics table
AUROC and AUCPR are threshold-free ranking metrics. Balanced Accuracy, F1 and MCC
are evaluated at each scoring function's published decision threshold, or at the data-driven
optimal threshold (Youden's J statistic) when no threshold was published (applies to Voronota only).
A row submitted via "Submit new predictions" below is instead evaluated with a declared, leakage-safe
cross-validation procedure (see that section) and is appended here, clearly labeled, rather than shown
in a separate table.
| Group | Participant | AUROC | AUCPR | Balanced Accuracy | F1 Score | MCC | Evaluation |
|---|---|---|---|---|---|---|---|
| Consensus (cross-group RF) | Random Forest consensus (top-221 features) | 0.947 | 0.945 | 0.883 | 0.884 | 0.765 | published threshold (resubstitution) |
| Bonvin | DeepRank-GNN | 0.860 | 0.816 | 0.804 | 0.823 | 0.618 | published threshold (resubstitution) |
| Schueler-Furman | Rosetta InterfaceAnalyzer | 0.849 | 0.821 | 0.792 | 0.802 | 0.586 | published threshold (resubstitution) |
| Guerois | IES (normalized) | 0.821 | 0.843 | 0.768 | 0.811 | 0.537 | published threshold (resubstitution) |
| Zou | DLScore-BC | 0.817 | 0.802 | 0.747 | 0.756 | 0.496 | published threshold (resubstitution) |
| SWISS-MODEL | QSQE | 0.808 | 0.918 | 0.724 | 0.741 | 0.396 | published threshold (resubstitution) |
| Venclovas | Voronota | 0.807 | 0.768 | 0.750 | 0.752 | 0.501 | data-driven optimal threshold (resubstitution) |
| Fernandez-Recio | NIPacking | 0.791 | 0.745 | 0.766 | 0.766 | 0.532 | published threshold (resubstitution) |
| Casadio (Bologna) | ISPRED4 | 0.769 | 0.747 | 0.720 | 0.716 | 0.440 | published threshold (resubstitution) |
| Correia | MaSIF | 0.766 | 0.747 | 0.708 | 0.732 | 0.417 | published threshold (resubstitution) |
| Kihara | GOAP | 0.754 | 0.713 | 0.693 | 0.708 | 0.387 | published threshold (resubstitution) |
| Oliva | CP/COCOMAPS | 0.749 | 0.712 | 0.693 | 0.695 | 0.386 | published threshold (resubstitution) |
| MOBI | shape_hydro_tails | 0.731 | 0.699 | 0.680 | 0.686 | 0.360 | published threshold (resubstitution) |
| Wolfson | ProbaConsensus (ScanNet F_Network) | 0.726 | 0.661 | 0.678 | 0.726 | 0.378 | published threshold (resubstitution) |
Quartile ranking
Each participant's per-metric quartile/cluster rank, computed independently for each
of the 5 performance measures over the current set of participants (higher value is better for all 5
metrics). Follows the same Classification Method currently selected in the chart above (Square
quartiles, Diagonal quartiles, or K-Means clustering): Square and Diagonal quartiles reduce to the same
percentile-based split for a single metric (the 2D "distance-to-corner" score diagonals uses is just the
metric's own normalized value when there is only one axis); K-Means runs real 1-D k-means (k=4) on each
metric's values independently, grouping by natural breaks in the data rather than equal population
percentiles, with clusters ranked by mean (Q1 = highest-performing cluster). Select "No classification"
to hide this table, matching the chart's own behavior. Recomputed automatically whenever a submission is
added or the Classification Method changes.
Select a classification method above
(Square quartiles, Diagonal quartiles, or K-Means clustering) to populate this table.
| Group | Participant | AUROC | AUCPR | Balanced Accuracy | F1 Score | MCC |
|---|
Datasets
| Dataset type | # instances |
|---|---|
| aggregation | 1 |
| assessment | 28 |
| input | 1 |
| metrics_reference | 1 |
| participant | 14 |
| public_reference | 1 |
Reference data used for evaluation:
gold_standard_labels.csv (physiological / crystallographic ground truth, 1671 entries) · cv_fold_assignment.csv (Strategy 1) · cv_fold_strategy2_mmseqs2_strict.csv (Strategy 2) · cv_fold_strategy2b_blastclust_reference.csv (Strategy 2b) · cv_fold_strategy3_combined_union.csv (Strategy 3, recommended) · full description of all 4 CV splitting strategies
gold_standard_labels.csv (physiological / crystallographic ground truth, 1671 entries) · cv_fold_assignment.csv (Strategy 1) · cv_fold_strategy2_mmseqs2_strict.csv (Strategy 2) · cv_fold_strategy2b_blastclust_reference.csv (Strategy 2b) · cv_fold_strategy3_combined_union.csv (Strategy 3, recommended) · full description of all 4 CV splitting strategies
Tools
| Group | Participant tool | Internal ID | Submission file |
|---|---|---|---|
| Casadio (Bologna) | ISPRED4 physiological-score (MCC-optimized) | 3DBioinfo:Bologna_ISPRED4 | predictions.csv ↓ |
| Bonvin | DeepRank-GNN | 3DBioinfo:Bonvin_DeepRank-GNN | predictions.csv ↓ |
| Correia | MaSIF descriptor-distance score | 3DBioinfo:Correia_MaSIF | predictions.csv ↓ |
| Fernandez-Recio | NIP (Normalized Interface Packing) | 3DBioinfo:FernandezRecio_NIPacking | predictions.csv ↓ |
| Schueler-Furman | Rosetta InterfaceAnalyzer (dG_cross/dSASA x100) | 3DBioinfo:Furman_InterfaceAnalyzer | predictions.csv ↓ |
| Guerois | IES homolog-normalized (10 seq, size-normalized) | 3DBioinfo:Guerois_IESnorm | predictions.csv ↓ |
| Kihara | GOAP statistical potential | 3DBioinfo:Kihara_GOAP | predictions.csv ↓ |
| MOBI | MOBI shape+hydrophobicity+tails integrated score | 3DBioinfo:MOBI_ShapeHydroTails | predictions.csv ↓ |
| Oliva | Contact-Propensity (CP) raw score | 3DBioinfo:Oliva_CP | predictions.csv ↓ |
| Consensus (cross-group RF) | Cross-validated Random Forest consensus classifier (top-221 features) | 3DBioinfo:RF_Consensus_top221 | predictions.csv ↓ |
| SWISS-MODEL | QSQE (Quaternary Structure Quality Estimate) | 3DBioinfo:SWISSMODEL_QSQE | predictions.csv ↓ |
| Venclovas | Voronota interface energy (normalized) | 3DBioinfo:Venclovas_Voronota | predictions.csv ↓ |
| Wolfson | Consensus classifier probability (SOAP+Network+FireDock) | 3DBioinfo:Wolfson_ProbaConsensus | predictions.csv ↓ |
| Zou | DLScore-BC | 3DBioinfo:Zou_DLScoreBC | predictions.csv ↓ |
Submit new predictions
Upload a scoring-function predictions file and see how it ranks against the 14
package participants, under a declared, leakage-safe cross-validation splitting procedure.
Every number shown here -- for the uploaded file AND for the 14 existing participants -- is a
held-out, cross-validated metric: for each fold of the chosen splitting strategy, the decision
threshold used by Balanced Accuracy / F1 / MCC is learned only from the other folds (Youden's J
statistic), never from the held-out fold itself, and no participant's published threshold is used in
this comparison. This is different from (and stricter than) the plain resubstitution comparison against
the manuscript shown in the Results section above.