ELIXIR 3DBioinfo Community PPI Benchmark Experimentlocal challenge preview
This is a local, offline preview rendered against this benchmark's own data, using the real open-source OpenEBench visualizer component -- it is not connected to openebench.bsc.es and does not reflect a live, published Benchmarking Event. Page organization follows the real OpenEBench benchmarking-event page structure (community → benchmarking event → challenges → leaderboard), referenced from the public CAID benchmarking page.

Community

ELIXIR 3D-BioInfo Community (3D-BioInfo)
ELIXIR 3D-BioInfo Community benchmarking activity on scoring and ranking of protein-protein interfaces and quaternary structure assignment, based on a curated benchmark of 1677 homodimer crystal structures.
protein-protein interactionquaternary structureinterface scoringhomodimersstructural bioinformaticsAlphaFold-multimer

Benchmarking Event

Benchmarking of scoring functions for physiological vs. crystallographic protein-protein interfaces
Created: 2023-05-15 · Modified: 2023-05-15

Challenges

PPI-PhysioClassification — Discrimination of physiological vs. non-physiological (crystallographic) homodimer interfaces

PPI-PhysioClassification challenge leaderboard

AUC-ROC vs. Accuracy-at-threshold · 14 participants
Community: 3D-BioInfo Challenge: PPI-PhysioClassification Metric X: AUC-ROC Metric Y: Accuracy@threshold

Full metrics table

AUROC and AUCPR are threshold-free ranking metrics. Balanced Accuracy, F1 and MCC are evaluated at each scoring function's published decision threshold, or at the data-driven optimal threshold (Youden's J statistic) when no threshold was published (applies to Voronota only). A row submitted via "Submit new predictions" below is instead evaluated with a declared, leakage-safe cross-validation procedure (see that section) and is appended here, clearly labeled, rather than shown in a separate table.
GroupParticipantAUROCAUCPRBalanced AccuracyF1 ScoreMCCEvaluation
Consensus (cross-group RF)Random Forest consensus (top-221 features)0.9470.9450.8830.8840.765published threshold (resubstitution)
BonvinDeepRank-GNN0.8600.8160.8040.8230.618published threshold (resubstitution)
Schueler-FurmanRosetta InterfaceAnalyzer0.8490.8210.7920.8020.586published threshold (resubstitution)
GueroisIES (normalized)0.8210.8430.7680.8110.537published threshold (resubstitution)
ZouDLScore-BC0.8170.8020.7470.7560.496published threshold (resubstitution)
SWISS-MODELQSQE0.8080.9180.7240.7410.396published threshold (resubstitution)
VenclovasVoronota0.8070.7680.7500.7520.501data-driven optimal threshold (resubstitution)
Fernandez-RecioNIPacking0.7910.7450.7660.7660.532published threshold (resubstitution)
Casadio (Bologna)ISPRED40.7690.7470.7200.7160.440published threshold (resubstitution)
CorreiaMaSIF0.7660.7470.7080.7320.417published threshold (resubstitution)
KiharaGOAP0.7540.7130.6930.7080.387published threshold (resubstitution)
OlivaCP/COCOMAPS0.7490.7120.6930.6950.386published threshold (resubstitution)
MOBIshape_hydro_tails0.7310.6990.6800.6860.360published threshold (resubstitution)
WolfsonProbaConsensus (ScanNet F_Network)0.7260.6610.6780.7260.378published threshold (resubstitution)

Quartile ranking

Each participant's per-metric quartile/cluster rank, computed independently for each of the 5 performance measures over the current set of participants (higher value is better for all 5 metrics). Follows the same Classification Method currently selected in the chart above (Square quartiles, Diagonal quartiles, or K-Means clustering): Square and Diagonal quartiles reduce to the same percentile-based split for a single metric (the 2D "distance-to-corner" score diagonals uses is just the metric's own normalized value when there is only one axis); K-Means runs real 1-D k-means (k=4) on each metric's values independently, grouping by natural breaks in the data rather than equal population percentiles, with clusters ranked by mean (Q1 = highest-performing cluster). Select "No classification" to hide this table, matching the chart's own behavior. Recomputed automatically whenever a submission is added or the Classification Method changes.
Select a classification method above (Square quartiles, Diagonal quartiles, or K-Means clustering) to populate this table.
GroupParticipantAUROCAUCPRBalanced AccuracyF1 ScoreMCC

Datasets

Dataset type# instances
aggregation1
assessment28
input1
metrics_reference1
participant14
public_reference1
Reference data used for evaluation:
gold_standard_labels.csv (physiological / crystallographic ground truth, 1671 entries) · cv_fold_assignment.csv (Strategy 1) · cv_fold_strategy2_mmseqs2_strict.csv (Strategy 2) · cv_fold_strategy2b_blastclust_reference.csv (Strategy 2b) · cv_fold_strategy3_combined_union.csv (Strategy 3, recommended) · full description of all 4 CV splitting strategies

Tools

GroupParticipant toolInternal IDSubmission file
Casadio (Bologna)ISPRED4 physiological-score (MCC-optimized)3DBioinfo:Bologna_ISPRED4predictions.csv ↓
BonvinDeepRank-GNN3DBioinfo:Bonvin_DeepRank-GNNpredictions.csv ↓
CorreiaMaSIF descriptor-distance score3DBioinfo:Correia_MaSIFpredictions.csv ↓
Fernandez-RecioNIP (Normalized Interface Packing)3DBioinfo:FernandezRecio_NIPackingpredictions.csv ↓
Schueler-FurmanRosetta InterfaceAnalyzer (dG_cross/dSASA x100)3DBioinfo:Furman_InterfaceAnalyzerpredictions.csv ↓
GueroisIES homolog-normalized (10 seq, size-normalized)3DBioinfo:Guerois_IESnormpredictions.csv ↓
KiharaGOAP statistical potential3DBioinfo:Kihara_GOAPpredictions.csv ↓
MOBIMOBI shape+hydrophobicity+tails integrated score3DBioinfo:MOBI_ShapeHydroTailspredictions.csv ↓
OlivaContact-Propensity (CP) raw score3DBioinfo:Oliva_CPpredictions.csv ↓
Consensus (cross-group RF)Cross-validated Random Forest consensus classifier (top-221 features)3DBioinfo:RF_Consensus_top221predictions.csv ↓
SWISS-MODELQSQE (Quaternary Structure Quality Estimate)3DBioinfo:SWISSMODEL_QSQEpredictions.csv ↓
VenclovasVoronota interface energy (normalized)3DBioinfo:Venclovas_Voronotapredictions.csv ↓
WolfsonConsensus classifier probability (SOAP+Network+FireDock)3DBioinfo:Wolfson_ProbaConsensuspredictions.csv ↓
ZouDLScore-BC3DBioinfo:Zou_DLScoreBCpredictions.csv ↓

Submit new predictions

Upload a scoring-function predictions file and see how it ranks against the 14 package participants, under a declared, leakage-safe cross-validation splitting procedure.
Every number shown here -- for the uploaded file AND for the 14 existing participants -- is a held-out, cross-validated metric: for each fold of the chosen splitting strategy, the decision threshold used by Balanced Accuracy / F1 / MCC is learned only from the other folds (Youden's J statistic), never from the held-out fold itself, and no participant's published threshold is used in this comparison. This is different from (and stricter than) the plain resubstitution comparison against the manuscript shown in the Results section above.
Columns: entry_id (e.g. 1a17_6) and a score column named score, raw_score or raw_score_oriented -- higher value must mean MORE physiological. download a template ↓