Reproducibility by Construction: The reproducible() Context and Run Metadata

Reproducibility is usually bolted on after a result looks good — and that is exactly when the seeds, library versions, and determinism flags have already been lost. The better approach is to make a run reproducible by construction: wrap the experiment in something that seeds every RNG and records the environment before any training happens. TGraphX offers this as a context manager in tgraphx/ux/reproducible_ctx.py. This tutorial shows how to use it and is candid about what determinism it can and cannot guarantee.

It is the hands-on companion to GNN Research Reproducibility and Seeded, Deterministic Experiments.

One context, many seeds

The reproducible(seed, deterministic=True) context manager does two jobs: it calls set_seed to seed Python, NumPy, and PyTorch RNGs (and configure cuDNN deterministic mode), and it records a snapshot of the reproducibility state. A lighter seeded(seed) alias runs the same thing with deterministic=False for when you want repeatable seeding without the full determinism cost.

import tgraphx as tgx
        
        with tgx.reproducible(seed=42, deterministic=True) as state:
            # build graph, train, evaluate — all RNGs seeded
            ...
        print(state)            # versions, platform, deterministic, warn_only
        print(tgx.reproducibility_state())   # inspect current state any time
        

The state dict captures the things you need to reproduce or debug a run later: Python/NumPy/PyTorch versions, platform, and the determinism flags in effect. Recording that with the result is what turns "it worked on my machine" into "here is exactly the environment it worked in."

Pair it with run metadata and serialized inputs

A reproducible context is most powerful alongside the artifact tooling. The tgraphx.tracking writers — write_run_metadata, write_dataset_metadata, and friends — persist the configuration of a run as JSON, and save_tgraphx (see Serializing Tensor Graphs) preserves the exact input graph including its mask split. Together they form a recoverable record:

With all four, re-running an experiment is a matter of restoring the seed, the environment, and the input — not reconstructing them from memory.

What it honestly cannot promise

This is the part most reproducibility guides skip, and it is the most important. The source is explicit about two limits:

1. It is one-way. reproducible() configures seeds and backend flags but does not roll them back on exit — the docstring says so directly. It is a "set up the environment" call, not a scoped sandbox; use reproducibility_state() to inspect what is currently in effect rather than assuming prior settings were restored.
2. CUDA determinism is partial. Some GPU operations have no deterministic implementation. The warn_only=True option lets such ops run with a warning instead of erroring, which is the same trade-off the PyTorch randomness notes describe. So exact bitwise reproducibility on GPU is not always achievable, and seeded() (deterministic=False) is offered precisely for when you accept that.

A third, universal caveat: determinism does not mean portability. Identical seeds can still produce different numbers across different hardware, CUDA versions, or library builds. Reproducibility tooling reduces variance and records context; it does not make floating-point arithmetic hardware-independent.

A by-construction checklist

1. Open a reproducible(seed=...) context at the top of the run.
2. Record state and call write_run_metadata(...).
3. save_tgraphx(...) the exact input graph.
4. On GPU, decide consciously between full deterministic=True (may error on non-deterministic ops unless warn_only) and seeded().
5. Note the package and CUDA versions — determinism is within an environment, not across environments.

The cross-machine reality

It is worth dwelling on the limit that surprises people most: a seed makes a run repeatable on the same machine, not identical across machines. Floating-point addition is not associative, GPUs schedule reductions non-deterministically unless forced, and different CUDA or library versions can change the last bits of a result. So two researchers with the same seed and code can still see slightly different numbers on different hardware — and that is expected, not a bug. What reproducible() and the recorded metadata buy you is the ability to explain a difference: with the versions and platform on hand, you can tell whether a discrepancy comes from a code change or merely from a different environment.

The practical stance: aim for exact reproducibility within an environment (fix the seed, enable deterministic mode where the ops support it), and aim for statistical reproducibility across environments (report mean and spread over several seeds, not a single number). Recording the environment is what makes the second kind honest. Determinism tooling reduces and explains variance; it does not abolish the arithmetic of floating point.

One more practical habit: write the reproducible() state dict into your run directory as JSON next to the results, so the seed and environment travel with the numbers rather than living only in a terminal you will eventually close. A result whose environment is recorded beside it is one you can defend months later.

Related guides

• Research Reproducibility in GNN Projects
• Seeded, Deterministic GNN Experiments
• Serializing Tensor Graphs

Conclusion

Reproducibility is most reliable when it is structural: tgx.reproducible(seed=...) seeds every RNG and records the environment up front, and pairing it with run metadata and save_tgraphx gives a recoverable record of a run. Just hold the honest limits in mind — it is a one-way setup call, GPU determinism is partial, and seeds do not erase hardware differences. Within those bounds, building reproducibility in from the first line is far more trustworthy than reconstructing it afterward.