Explaining Tensor GNNs: Saliency, Integrated Gradients, and Edge Attribution

A prediction you cannot interrogate is hard to trust, and graph models add two attribution questions at once: which parts of a node's features mattered and which edges mattered. When nodes are [C, H, W] tensors, the first question becomes spatial — which pixels of which patch drove the decision. TGraphX ships an explainability module, tgraphx/explain, that answers both with standard, well-understood attribution methods. This note surveys what is available and, importantly, how to read attributions responsibly.

It connects to tensor graph attention (attention is one attribution signal among several) and to multi-modal nodes (where spatial attributions become image heatmaps).

Node feature attribution

The module exposes two complementary node-attribution methods:

• node_feature_saliency — the gradient of the output with respect to node features. Large-magnitude gradients flag inputs the prediction is locally sensitive to. It is cheap (one backward pass) but, as with all raw-gradient methods, can be noisy.
• integrated_gradients — the path-integral attribution of Sundararajan et al., which integrates gradients along a path from a baseline to the input. It satisfies useful axioms (sensitivity, completeness) that raw saliency does not, at the cost of several forward/backward passes.

Because node features are tensors, these attributions have the same [C, H, W] shape as the input — an attribution map, not a single number per node. That is what lets the patch-heatmap helpers (below) render them spatially.

Edge attribution

Two methods attribute the prediction to edges rather than nodes:

The two answer subtly different questions. Gradient attribution asks "how sensitive is the output to this edge right now?"; perturbation attribution asks "what happens if this edge is not there?" They often agree, and disagreement is itself informative. Separately, attention_to_edge_scores (in tgraphx/explain/attention.py) exposes a TensorGATLayer's learned attention as an edge signal — but note the caution below.

Rendering spatial attributions

For tensor nodes, an attribution is most useful when you can see it. patch_saliency_to_image_grid and patch_saliency_to_volume_projection (in tgraphx/explain/patch_heatmap.py) map per-patch saliency back onto the original image grid or a volume projection, so a [C, H, W] attribution over patch nodes becomes a heatmap you can overlay on the source image. This closes the loop with the multi-modal node construction: patches went in as nodes, and attributions come back out as a spatial map.

Reading attributions responsibly

This is the section that keeps explainability honest. Attribution methods explain sensitivity, not causation: a high saliency means the output is locally sensitive to an input, not that the input "caused" the decision in any deeper sense. Raw saliency is noisy and can be misleading; integrated gradients is better-behaved but depends on the baseline you choose. And a crucial caveat about attention: attention weights are not explanations — there is a well-known literature showing attention can be high on features that do not drive the output, so attention_to_edge_scores should be treated as one signal to corroborate, not as ground truth. TGraphX provides the tools; interpreting them soundly is the user's responsibility, and cross-checking gradient, perturbation, and attention signals is the conservative practice.

Honest framing

The explainability subsystem is Beta — tested and documented, API stable within v1.x. It implements standard methods faithfully rather than proposing new ones, which is the right scope: established, auditable attributions you can reason about. It does not claim to produce causal or human-final explanations, and you should report which method and baseline you used, since attributions are method-dependent.

A corroboration workflow

Because no single attribution is ground truth, the trustworthy practice is corroboration. A workflow that catches misleading explanations: compute node_feature_saliency for a fast first look, then integrated_gradients with a sensible baseline (often a zero or mean input) for an axiomatically better attribution, and check whether the two broadly agree — large disagreement is a signal to distrust the cheap one. For structure, run both edge_gradient_attribution and edge_perturbation_attribution; if the gradient says an edge matters but removing it barely changes the output, the gradient is reading local sensitivity that does not translate to importance.

Treat any attention-derived signal as the weakest of the three and never as a standalone explanation. Finally, sanity-check against a baseline you trust: does the attribution highlight inputs a domain expert would expect, and does perturbing a "highly attributed" region actually degrade the prediction? Attribution methods are most useful when you triangulate across them and validate against a perturbation you can measure, rather than reporting a single colourful heatmap as the explanation.

As a closing discipline, always report the attribution method and its baseline, and state plainly that it measures sensitivity rather than cause. An explanation is only trustworthy when the reader knows how it was produced and what it deliberately does not claim — anything less invites over-interpretation of a colourful map.

Related guides

• Graph Attention over Feature Maps
• Multi-Modal Nodes: Images, Volumes, and Sequences
• Inside Tensor Message Passing

Conclusion

TGraphX explains tensor GNNs at two levels: node-feature attribution (node_feature_saliency, integrated_gradients) that yields spatial maps you can render as patch heatmaps, and edge attribution (gradient and perturbation) that ranks relations. The methods are standard and Beta-stable; the discipline is in reading them as sensitivity, not causation, and corroborating across methods rather than trusting attention alone. Used that way, they make a tensor GNN's decisions inspectable instead of opaque.