Interpreting Your Results
After cpm.run(...) finishes, everything CCCPM produces is written to the
results_directory you specified. This page explains what each output means and
how to read it — both the interactive HTML report and the underlying CSV/array
files you can load for your own figures and statistics.
The same structure is produced for regression and classification; only the performance metrics differ (see Metrics).
Two concepts that appear everywhere
Almost every result is broken down by a model and a network.
Models — what each one is built from
For every cross-validation fold, CCCPM fits several models so you can see how much the connectome actually adds on top of your covariates:
| Model | Predicts the target from… | Use it to answer… |
|---|---|---|
connectome |
connectome network strength only | How well does brain connectivity alone predict? |
covariates |
the covariates only (e.g. age, sex, motion) | What's the baseline from nuisance variables? |
full |
connectome + covariates | Best combined prediction |
residuals |
connectome, after removing covariate effects from the target | Connectome contribution with confounds removed |
increment |
the added value of the connectome over covariates alone (full − covariates) |
Does the connectome explain anything beyond confounds? |
The increment model is usually the most important for a confound-aware claim: a
significant increment means the connectome carries predictive information that the
covariates do not.
Networks — which edges are used
Edge selection splits predictive edges into two sub-networks, and CCCPM evaluates each plus their combination:
- positive — edges positively correlated with the target.
- negative — edges negatively correlated with the target.
- both — positive and negative networks combined.
So a single number like "Pearson r" is always reported per (model, network) pair.
The HTML report (report.html)
report.html is a self-contained, shareable summary — open it in any browser, print it
to PDF, or send it to a collaborator (everything, including the figures, is embedded, so
it works offline). It is a single scrolling page with a sticky sidebar, organised top-down
from the headline result to the supporting detail:
| Section | What it shows |
|---|---|
| Summary | The headline cross-validated result as a one-sentence verdict, key-stat chips (samples, nodes, edges, covariates, permutations, edge p-threshold), and predicted-vs-observed scatter plots for the positive, negative, and both networks plus the covariates-only baseline. Each scatter is annotated with its cross-validated effect size (Pearson r / AUC) and permutation p. Start here. |
| Model Comparison | One faceted figure comparing every model (connectome, covariates, full, residuals, increment) across metrics and networks, plus the APA results table (mean [SD] with permutation p-values). Foregrounds the increment model — your confound-control evidence. |
| Network Strengths | How the summed positive/negative network strength relates to the target, and the distribution of strength across participants. |
| Brain & Edges | The predictive edges in the brain: a connectivity matrix, a hub (node-degree) plot, and — when an atlas with a network column is supplied — a network-summary matrix and a chord diagram. With node coordinates (x, y, z) it also renders a glass-brain view. These figures come from the atlas you pass to CPMAnalysis. |
| Stable Edges | All significant edges (region A — region B) per network, the significance method and its diagnostics (e.g. largest NBS component), the permutation null-distribution plots, and a downloadable CSV of every selected edge with its stability and significance. |
| Data & Methods | Appendix: target distribution, covariate scatter matrix, the full run configuration, and (with an inner CV) per-fold hyperparameters. |
Every section carries a short always-visible "what this shows" note so a reader unfamiliar with the internals can follow it.
You can view a full example report in the Simulated Data example.
Metrics
Metrics are computed out-of-sample (on held-out test folds) and then averaged across folds. Higher is better unless noted.
Regression
| Metric | Meaning | Notes |
|---|---|---|
pearson_score |
Correlation between predicted and true values | The conventional CPM headline number |
explained_variance_score |
Fraction of target variance explained | 1.0 is perfect; can be negative |
mean_absolute_error |
Average absolute prediction error | Lower is better; in target units |
mean_squared_error |
Average squared prediction error | Lower is better; penalises large errors |
Classification
| Metric | Meaning | Notes |
|---|---|---|
roc_auc |
Area under the ROC curve | Threshold-independent; robust headline metric |
balanced_accuracy |
Mean of sensitivity and specificity | Prefer over accuracy for imbalanced classes |
accuracy |
Fraction correctly classified | Misleading when classes are imbalanced |
f1_score |
Harmonic mean of precision and recall | Useful when the positive class is rare |
Output files
Everything in the report is also available as plain files for your own analysis.
Performance
| File | Contents |
|---|---|
cv_results_summary.csv |
Mean ± std of every metric, indexed by (model, network). The numbers behind the Main Results page. |
cv_results_full.csv |
Per-fold metric values (before averaging) — use these for your own error bars / tests. |
cv_predictions.csv |
Out-of-sample predictions per subject: columns sample_index, model, network, y_pred, y_true, fold. |
cv_network_strengths.csv |
Summed positive/negative network strength per subject, with the target: y_true, network_strength, model, network, fold. |
Significance (only if n_permutations > 0)
| File | Contents |
|---|---|
p_values.csv |
Permutation p-value for each (network, model) × metric. A small value (e.g. ≤ 0.05) means the observed performance is unlikely under the null of no brain–behaviour association. |
permutation/ |
The full null distributions (cv_results_*) and permuted edge selections used to compute those p-values. |
How the p-values are defined: the entire pipeline is re-run on many shuffled
copies of the target to build a null distribution of performance; the p-value is the
fraction of permutations that match or beat the real result. With n_permutations=100
the smallest possible p-value is ~0.01 — use 1000+ permutations for publishable
significance.
Selected edges & stability
| File | Contents |
|---|---|
edges.npy |
The edges selected in each fold. |
stability_edges.npy |
How consistently each edge was selected across folds (its stability). Edges chosen in most folds are the reliable ones. |
stability_edges_significance.npy |
Per-edge significance of edge stability (when permutations are run). With the default NBS method these are subnetwork-level p-values shared by every edge in a significant connected component; with TFCE they are per-edge FWER-corrected p-values. |
stability_edges_significance_meta.json |
Diagnostics for the significance test: method + parameters, the permutation null distribution, and (for NBS) the observed components and largest-component size. Drives the null-distribution plots in the report. |
data_insights/ |
Cached input summaries and figures (target distribution, covariate scatter, feature/target/covariate names). |
Logs
cpm_log.txt— a full, timestamped log of the run (configuration, per-step progress, final aggregated table). The first place to look if something behaved unexpectedly.task_type.txt—regressionorclassification, recorded for the report.plots/— every figure from the report exported as PNG/PDF/SVG for direct use in papers.
A reading workflow
- Open
report.html→ Summary for the headline result, then Model Comparison. Look at thefullandincrementmodels on thebothnetwork. - Check
p_values.csv: is theincrementmodel significant? That's your evidence the connectome adds information beyond confounds. - Inspect
stability_edges.npy(or the Stable Edges section) to see which connections drive the prediction. - Use
cv_predictions.csv/cv_results_full.csvto make your own figures or run additional statistics.