Applying TGraphX to Citation Networks

Citation networks are the most-studied benchmark domain in graph machine learning. Datasets like Cora, Citeseer, and PubMed have been used to evaluate nearly every major GNN architecture since GCN was proposed in 2017. Understanding why these graphs work so well for evaluation — and what their structure actually looks like — is essential context for anyone building or benchmarking GNNs. This article walks through a complete TGraphX workflow on a citation network, from loading raw data to training a model and evaluating it properly.

What This Builds On

This tutorial assumes you have TGraphX installed and understand the basic Graph object and message-passing layers. The shape-aware validation guide is useful for avoiding silent shape errors when building the feature matrix. For context on reproducibility and consistent evaluation, see the GNN research reproducibility article.

Citation Networks as Graphs

In a citation network, nodes represent academic papers and edges represent citation relationships. Each node has a feature vector (typically a bag-of-words or TF-IDF representation of the paper's abstract or title words) and a class label (the research area or topic of the paper).

The Cora dataset has 2,708 nodes (papers), 5,429 edges (citations), 1,433-dimensional binary bag-of-words features, and 7 classes. Citeseer has 3,327 nodes, 4,732 edges, 3,703-dimensional features, and 6 classes. PubMed has 19,717 nodes, 44,338 edges, 500-dimensional TF-IDF features, and 3 classes.

These graphs exhibit strong homophily: papers that cite each other tend to be in the same research area. The homophily ratio (fraction of edges connecting same-class nodes) is roughly 0.81 for Cora, 0.74 for Citeseer, and 0.80 for PubMed. This makes them favorable for GNNs that average over neighborhoods — the averaging tends to reinforce the correct class signal rather than dilute it.

Loading Citation Network Data

We will construct a minimal citation network from scratch for this tutorial, rather than relying on a specific dataset loading library. In practice, you would download Cora or a similar dataset and parse its node features and adjacency information.

import torch
        import numpy as np
        from tgraphx import Graph
        
        def load_cora_format(node_feature_file, edge_file, label_file):
            """
            Load citation network from common file format.
            node_feature_file: each row is 'node_id feat1 feat2 ... label'
            edge_file: each row is 'source_id target_id'
            """
            # Load node features and labels
            data = np.loadtxt(node_feature_file)
            x = torch.tensor(data[:, :-1], dtype=torch.float)
            y = torch.tensor(data[:, -1], dtype=torch.long)
        
            # Load edges
            edges = np.loadtxt(edge_file, dtype=np.int64)
            edge_index = torch.tensor(edges.T, dtype=torch.long)
        
            return Graph(node_features=x, edge_index=edge_index, node_labels=y)
        
        # For this tutorial, create a synthetic graph with Cora-like properties
        torch.manual_seed(42)
        N, D, C = 500, 64, 7   # 500 nodes, 64 features, 7 classes
        
        # Create features and labels with homophily bias
        y = torch.randint(0, C, (N,))
        x = torch.randn(N, D)
        # Add class-correlated signal to features
        class_centers = torch.randn(C, D)
        x += class_centers[y] * 0.5   # nodes in the same class have correlated features
        
        # Create edges with homophily bias (same-class nodes are more likely connected)
        edges_src, edges_dst = [], []
        for i in range(N):
            # Higher probability of connecting to same-class nodes
            same_class = (y == y[i]).nonzero(as_tuple=True)[0]
            diff_class = (y != y[i]).nonzero(as_tuple=True)[0]
            k_same = min(3, same_class.numel())
            k_diff = min(1, diff_class.numel())
            if k_same > 0:
                targets = same_class[torch.randperm(same_class.numel())[:k_same]]
                for t in targets:
                    if t.item() != i:
                        edges_src.append(i); edges_dst.append(t.item())
            if k_diff > 0:
                targets = diff_class[torch.randperm(diff_class.numel())[:k_diff]]
                for t in targets:
                    edges_src.append(i); edges_dst.append(t.item())
        
        edge_index = torch.tensor([edges_src, edges_dst], dtype=torch.long)
        g = Graph(node_features=x, edge_index=edge_index, node_labels=y)
        print(f"Graph: {N} nodes, {edge_index.size(1)} edges, {C} classes")
        

Standard Train/Val/Test Split

The standard Cora split uses 20 labeled nodes per class for training (140 total), 500 nodes for validation, and 1,000 nodes for testing — leaving most nodes unlabeled. This is an extremely small labeled set relative to graph size, which is why citation network benchmarks are considered semi-supervised.

from tgraphx.estimators.splits import random_node_split
        
        # Use the standard proportions from the original GCN paper
        # 20 per class for training (~140 total), 500 val, 1000 test
        train_mask = torch.zeros(N, dtype=torch.bool)
        val_mask = torch.zeros(N, dtype=torch.bool)
        test_mask = torch.zeros(N, dtype=torch.bool)
        
        # 20 nodes per class for training
        for c in range(C):
            class_nodes = (y == c).nonzero(as_tuple=True)[0]
            if class_nodes.numel() >= 20:
                selected = class_nodes[torch.randperm(class_nodes.numel())[:20]]
                train_mask[selected] = True
        
        # Remaining nodes: 500 val, 1000 test
        remaining = (~train_mask).nonzero(as_tuple=True)[0]
        remaining = remaining[torch.randperm(remaining.numel())]
        val_mask[remaining[:50]] = True    # scaled down for our 500-node example
        test_mask[remaining[50:150]] = True
        
        print(f"Train: {train_mask.sum()}, Val: {val_mask.sum()}, Test: {test_mask.sum()}")
        

Building the GNN Model

For citation networks, a two-layer GNN with dropout is the standard baseline. TGraphX's GraphSAGE layer works well here:

import torch.nn as nn
        import torch.nn.functional as F
        from tgraphx.layers.sage import TensorGraphSAGELayer
        
        class CitationGNN(nn.Module):
            def __init__(self, in_dim, hidden_dim, num_classes, dropout=0.5):
                super().__init__()
                self.conv1 = TensorGraphSAGELayer(in_dim, hidden_dim)
                self.conv2 = TensorGraphSAGELayer(hidden_dim, num_classes)
                self.dropout = dropout
        
            def forward(self, x, edge_index):
                x = F.relu(self.conv1(x, edge_index))
                x = F.dropout(x, p=self.dropout, training=self.training)
                return self.conv2(x, edge_index)
        
        model = CitationGNN(in_dim=D, hidden_dim=64, num_classes=C)
        print(f"Model parameters: {sum(p.numel() for p in model.parameters()):,}")
        

Training Loop with Early Stopping

from tgraphx.estimators.early_stopping import EarlyStopping
        from tgraphx.reproducibility import set_reproducibility
        
        with set_reproducibility(seed=42):
            model = CitationGNN(in_dim=D, hidden_dim=64, num_classes=C, dropout=0.5)
            optimizer = torch.optim.Adam(model.parameters(), lr=5e-3, weight_decay=5e-4)
            stopper = EarlyStopping(patience=50, min_delta=1e-4)
        
            best_val_acc = 0.0
        
            for epoch in range(500):
                model.train()
                logits = model(g.node_features, g.edge_index)
                loss = F.cross_entropy(logits[train_mask], y[train_mask])
                optimizer.zero_grad()
                loss.backward()
                optimizer.step()
        
                model.eval()
                with torch.no_grad():
                    val_logits = model(g.node_features, g.edge_index)
                    val_loss = F.cross_entropy(val_logits[val_mask], y[val_mask])
                    val_preds = val_logits[val_mask].argmax(dim=1)
                    val_acc = (val_preds == y[val_mask]).float().mean().item()
        
                if val_acc > best_val_acc:
                    best_val_acc = val_acc
                    best_state = {k: v.clone() for k, v in model.state_dict().items()}
        
                if stopper.step(val_loss.item()):
                    print(f"Early stopping at epoch {epoch}")
                    break
        
            # Load best checkpoint
            model.load_state_dict(best_state)
        

Evaluation Best Practices

Evaluating on citation networks requires several careful choices:

Use the test set only once. A common mistake is to run evaluation on the test set many times during development and tune hyperparameters based on test performance. This inflates reported accuracy. Always tune on the validation set and report test accuracy exactly once at the end.

Report multiple runs. Citation network splits are small enough that random seed variance is significant. The standard practice is to report mean and standard deviation across 10 or more runs with different random seeds.

Do not cherry-pick seeds. If you run 50 experiments and report the best single result, you are reporting an upper confidence bound, not a realistic performance estimate.

# Final evaluation
        model.eval()
        with torch.no_grad():
            test_logits = model(g.node_features, g.edge_index)
            test_preds = test_logits[test_mask].argmax(dim=1)
            test_acc = (test_preds == y[test_mask]).float().mean().item()
        
        print(f"Test accuracy: {test_acc:.4f}")
        print(f"Best val accuracy: {best_val_acc:.4f}")
        

Homophily and When GNNs Struggle

The strong homophily of Cora and Citeseer is both their appeal and their limitation as benchmarks. A GNN trained and evaluated on Cora is being tested in a setting that strongly favors message-passing aggregation — neighboring nodes are likely in the same class, so averaging their features reinforces the correct label.

Real-world graphs often have weaker homophily. E-commerce graphs, fraud detection graphs, and protein interaction networks may have homophily ratios below 0.3. GNNs that work well on Cora may fail badly on these graphs.

When evaluating on citation networks, consider computing the homophily ratio of your dataset:

def homophily_ratio(edge_index, labels):
            """Fraction of edges connecting same-class nodes."""
            src, dst = edge_index
            same_class = (labels[src] == labels[dst]).float()
            return same_class.mean().item()
        
        h = homophily_ratio(g.edge_index, y)
        print(f"Homophily ratio: {h:.4f}")
        

A ratio above 0.5 indicates that GNN averaging will generally help. Below 0.5, aggregation may hurt and alternative architectures (higher-order methods, feature-augmented GNNs, MLP baselines) are worth considering.

Adding Node2Vec Features

For citation networks with sparse or weak text features, Node2Vec embeddings derived from graph structure alone can supplement or replace the original features. TGraphX's tgraphx.mining.node2vec module provides this:

from tgraphx.mining.node2vec import Node2Vec
        
        # Train Node2Vec embeddings on the citation graph structure
        node2vec = Node2Vec(
            edge_index=g.edge_index,
            num_nodes=N,
            embedding_dim=64,
            walk_length=20,
            context_size=10,
            walks_per_node=10,
            p=1.0,     # return parameter (BFS vs DFS bias)
            q=1.0,     # in-out parameter
        )
        
        node2vec.train(epochs=50, batch_size=128, lr=0.01)
        structural_embeddings = node2vec.get_embeddings()   # [N, 64]
        
        # Concatenate with original features
        x_augmented = torch.cat([g.node_features, structural_embeddings], dim=1)
        

Node2Vec embeddings capture structural roles: nodes in similar positions within the graph (hub nodes, peripheral nodes, bridge nodes) will have similar embeddings regardless of their actual connections. This complements text-based features, which capture semantic similarity.

For the standard Cora/Citeseer benchmarks, adding Node2Vec features often provides a small but consistent improvement, particularly when training with very few labeled nodes per class.

Anomaly Detection in Citation Networks

Not all papers in a citation network belong to a clean category. Retracted papers, cross-disciplinary work, and survey papers can be structural outliers. TGraphX's tgraphx.mining.anomaly module provides anomaly detection tools that identify nodes whose neighborhoods are inconsistent with their features:

from tgraphx.mining.anomaly import AnomalyDetector
        
        detector = AnomalyDetector(method='feature_distance')
        anomaly_scores = detector.fit_predict(g)
        
        # Nodes with high scores are structural outliers
        top_anomalies = anomaly_scores.topk(10).indices
        print("Top anomalous nodes:", top_anomalies.tolist())
        

Inspecting anomalies before training can improve model quality by identifying mislabeled nodes or nodes that violate the homophily assumption used by GNN aggregation.

Limitations and Honest Notes

Citation networks are heavily overfit benchmarks. Dozens of GNN architectures have been developed and tuned specifically on Cora, Citeseer, and PubMed. Marginal improvements on these datasets do not necessarily generalize to other graph learning tasks. Treat citation network results as a sanity check, not a definitive performance measure.

The standard 140/500/1000 split for Cora is non-random and has been used so many times that many models have been tuned to this specific split. Results on this split are not representative of performance on different splits of the same graph. Use random splits with multiple seeds for a more honest evaluation.

The node features in real Cora are binary bag-of-words vectors with many zeros, which is quite different from the continuous Gaussian features used in this tutorial. Real citation networks also have a much larger vocabulary dimension (1,433 for Cora) which affects the relative importance of structural vs feature-based classification.

The GNN research reproducibility article covers additional best practices for reporting results, including how to handle random seeds, which metrics to report, and how to compare against baselines fairly.

Frequently Asked Questions

Should I use the full graph for training or only the training nodes?
With transductive node classification (the standard citation network setup), the full graph's edge structure is available during training. You train on the labels of the training nodes but use all edges for message passing. This is the correct setup — withholding edges from the GNN during training would hurt performance artificially.

What accuracy is considered good on Cora?
Published results range from about 81% (GCN, 2017) to around 85–87% for more recent methods. However, these numbers are specific to the original split and are not directly comparable to results on random splits. On random splits with multiple seeds, variance is significant and the "good" threshold is lower.

Can TGraphX reproduce results from published GCN papers?
TGraphX provides the building blocks (message-passing layers, dropout, early stopping) but does not include pre-tuned hyperparameters for standard benchmarks. Reproducing published numbers requires carefully matching the original training setup, including the specific split, optimizer settings, and number of hidden units.