Designing a Graph Experiment Report That Future You Can Understand
Six months after an experiment, "future you" is effectively a new person. The
context that seemed obvious — why you chose k=5 for the k-NN graph, which
dataset split you used, what hardware the numbers came from — has evaporated.
A well-designed experiment report makes the experiment legible without access
to working memory.
This tutorial builds a concrete report template for graph learning experiments,
using TGraphX's write functions to automate the structural parts.
What a Graph Experiment Report Needs to Contain
Unlike standard neural network experiments, graph learning experiments have
additional context that must be recorded:
| Category | What to include |
|---|---|
| Graph schema | Node feature shape, edge count, edge construction |
| Topology summary | Node count, avg degree, connectivity properties |
| Tensor shapes | Exact shapes at each pipeline stage |
| Model architecture | Layers, dims, activations, regularization |
| Training config | Optimizer, lr, epochs, batch size, seed |
| Environment | Python, PyTorch, TGraphX versions, hardware |
| Results | Train/val/test metrics, confidence intervals if any |
| Caveats | Known limitations, hardware dependencies, assumptions |
Missing any of these categories makes future reconstruction unreliable.
Section 1: Graph Schema
The graph schema is the most GNN-specific section. It answers: "what kind of
graph did we use?"
markdown
## Graph Schema
- **Node features**: shape [N, 3, 16, 16] — image patches, 3 channels, 16×16
- **Edge index**: shape [2, E] — directed edges
- **Edge features**: shape [E, 1] — no meaningful edge features (zeros)
- **Node labels**: shape [N] — 5 classes (0-4)
- **Edge construction**: build_grid_graph(height=4, width=4) — grid connectivity
- **Graph count** (if batched dataset): 1200 training, 300 validation, 300 test
- **Nodes per graph** (mean ± std): 16.0 ± 0.0 (fixed grid size)
Use write_graph_stats to automate this:
python
from tgraphx import write_graph_stats
write_graph_stats(g, path="./report/graph_schema.json")
Section 2: Environment
markdown
## Environment
Generated by env_report(include_hardware=True):
- Python: 3.11.4
- PyTorch: 2.1.0+cu118
- TGraphX: 1.4.2 (Beta — pinned in requirements.txt)
- CUDA: 11.8
- GPU: NVIDIA RTX 3090 (24 GB)
- CPU: 16 cores
Note: Results may differ on different hardware due to float32 vs float16
differences and non-deterministic CUDA operations.
Automate with:
python
from tgraphx.performance import env_report
from tgraphx import write_hardware_report
info = env_report(include_hardware=True)
write_hardware_report(path="./report/hardware.json")
Section 3: Reproducibility Config
markdown
## Reproducibility
- Random seed: 42 (set_seed(42, deterministic=True))
- Deterministic mode: True (PyTorch deterministic algorithms enabled)
- Data split: random 80/10/10 with seed 42
- Shuffle in DataLoader: True, generator seeded with 42
Caveat: deterministic=True does not guarantee bit-for-bit reproduction
across different GPU hardware or different CUDA versions.
Section 4: Model Architecture
markdown
## Model Architecture
- Type: 2-layer GNN with TensorGraphSAGELayer
- Input dim: 768 (flattened 3×16×16 patch)
- Hidden dim: 128
- Output dim: 5 (num_classes)
- Activation: ReLU between layers
- Dropout: 0.1 (applied after each layer)
- Parameters: ~200K total
Use write_experiment_config for the machine-readable version:
python
from tgraphx import write_experiment_config
write_experiment_config({
"model_type": "2-layer TensorGraphSAGELayer",
"input_dim": 768,
"hidden_dim": 128,
"output_dim": 5,
"dropout": 0.1,
"activation": "ReLU",
"num_parameters": sum(p.numel() for p in model.parameters()),
}, path="./report/model_config.json")
Section 5: Training Configuration
markdown
## Training Configuration
- Optimizer: Adam(lr=0.001, weight_decay=1e-5)
- Learning rate schedule: None
- Epochs: 100 (early stopping with patience=15 on val_loss)
- Batch size: 32 graphs per batch (GraphDataLoader)
- Loss: CrossEntropyLoss
- Early stopping: activated at epoch 67 (val_loss did not improve for 15 epochs)
If using TGraphX's EarlyStopping and Runner:
python
from tgraphx.experiments import EarlyStopping, Runner, ExperimentConfig
from tgraphx import write_run_metadata
early_stop = EarlyStopping(patience=15, monitor="val_loss")
config = ExperimentConfig(model=model, optimizer=optimizer,
criterion=criterion, epochs=100)
runner = Runner(config, callbacks=[early_stop])
runner.fit(train_loader, val_loader)
write_run_metadata({
"early_stopped": early_stop.triggered,
"best_epoch": runner.best_epoch,
"total_epochs_run": runner.current_epoch,
}, path="./report/run_metadata.json")
Section 6: Results
markdown
## Results
| Metric | Train | Validation | Test |
|----------------------|--------|------------|--------|
| Accuracy | 0.923 | 0.847 | 0.851 |
| Loss (CrossEntropy) | 0.214 | 0.378 | 0.362 |
Evaluation: single run (N=1). Confidence intervals not computed.
OGBNodeEvaluator not applicable (custom dataset).
Note: Results are specific to the hardware and seed listed above.
## The Report Template
A minimal graph experiment report should answer these questions for a reader who was not present:
### Section 1: Data
| Field | Example value |
|---|---|
| Dataset name / source | Cora citation network, public |
| Number of nodes | 2708 |
| Number of edges | 10556 |
| Node feature shape | [2708, 1433] (bag-of-words) |
| Node label classes | 7 |
| Train/val/test split | 140 / 500 / 1000 |
| Preprocessing | Normalized adjacency, self-loops added |
```python
from tgraphx import write_graph_stats, write_dataset_metadata
write_graph_stats(g, path="runs/exp_001/graph_stats.json")
write_dataset_metadata({
"name": "Cora",
"source": "https://...",
"split": {"train": 140, "val": 500, "test": 1000},
"preprocessing": ["normalize_adjacency", "add_self_loops"],
}, path="runs/exp_001/dataset_metadata.json")
Section 2: Model
| Field | Value |
|---|---|
| Architecture | TensorGraphSAGE (2 layers) |
| Hidden channels | 64 |
| Aggregation | mean |
| Dropout | 0.5 |
| Parameters | ~180K |
Section 3: Training
python
from tgraphx import write_experiment_config
write_experiment_config({
"optimizer": "Adam",
"lr": 0.01,
"weight_decay": 5e-4,
"epochs": 200,
"batch_size": "full_graph",
"seed": 42,
"deterministic": True,
}, path="runs/exp_001/config.json")
Section 4: Hardware and Environment
python
from tgraphx import env_report, write_hardware_report
env = env_report(include_hardware=True)
write_hardware_report(env, path="runs/exp_001/hardware.json")
# Records: python, torch, tgraphx versions; CUDA availability; CPU/RAM
Section 5: Results
| Metric | Value | Notes |
|---|---|---|
| Test accuracy | 81.2% | Mean over 5 seeds |
| Test accuracy std | ±0.3% | |
| Best val epoch | 143 | |
| Training time | 42s | Single V100 |
Section 6: Caveats and Limitations
This section is the most commonly omitted and most valuable:
- Did you tune hyperparameters on the test set? (invalidates the number)
- Is the split standard or custom? (affects comparability)
- Did you run only one seed? (high variance for small datasets)
- Any known preprocessing issues?
Using the experiments Module
python
from tgraphx.experiments import Runner, load_config
cfg = load_config("configs/exp_001.yaml")
runner = Runner(cfg)
result = runner.fit()
# Runner saves metrics, config, and checkpoints automatically
Runner from tgraphx.experiments (v0.3.0) structures the training loop and saves artifacts to your run directory. Combine it with the write_* utilities for a complete reproducible record.
Report Template Diagram
runs/exp_001/
├── config.json ← hyperparams, seed, optimizer
├── dataset_metadata.json ← data source, split, preprocessing
├── graph_stats.json ← N, E, feature shapes
├── hardware.json ← python/torch/cuda versions + hardware
├── metrics.csv ← epoch-level training metrics
├── model_best.pt ← best checkpoint
└── README.md ← human notes, caveats, known issues
Six months from now, you will thank yourself for writing Section 6.
Save experiment context with results.