Building a Minimal Article-to-Graph Pipeline for NLP Experiments
Text, unlike images, doesn't have a natural grid structure. A sentence is
a sequence, but words relate to each other in ways that go beyond adjacency —
subject-verb relations, co-reference, thematic connections across paragraphs.
Graphs are a natural way to represent these relationships explicitly.
This tutorial builds a minimal pipeline from raw text to a Graph object
in TGraphX, covering two common edge construction strategies: co-occurrence
windows and semantic similarity. The focus is on the construction mechanics;
what you do with the graph downstream depends on your task.
Why Build a Text Graph?
Text graphs are useful when:
- You want to model relationships between entities across a document
- Sentence-level representations need to be combined into document-level
representations
- You're working on tasks like claim verification, document classification,
or information extraction where relational structure matters
Text graphs are probably not useful when:
- Sequence models (transformers) already handle the task well
- The graph structure doesn't add information not already captured by context
This is a practical design decision, not a claim about graph methods
outperforming transformers on standard NLP benchmarks.
Step 1: Text to Nodes
The first step is deciding what a node represents. Common choices:
| Node type | Granularity | Use case |
|---|---|---|
| Word tokens | Fine-grained | Syntactic analysis, NER |
| Sentences | Medium | Document classification, QA |
| Paragraphs | Coarse | Long-document understanding |
| Named entities | Task-dependent | Knowledge graph construction |
For this tutorial, sentences are nodes. Each sentence becomes a node with an
embedding as its feature.
import torch
# Example: 5-sentence document
sentences = [
"Graph neural networks extend deep learning to irregular data.",
"Nodes represent entities and edges represent relationships.",
"Message passing aggregates neighborhood information.",
"TGraphX provides tensor-valued node features.",
"Reproducibility requires saving seeds and configurations.",
]
# Sentence embeddings (in practice, use a sentence encoder)
# Here we simulate with random vectors for demonstration
d = 128
node_features = torch.randn(len(sentences), d) # [N=5, d=128]
print(f"node_features shape: {node_features.shape}")
In a real pipeline, replace torch.randn(...) with a sentence encoder output:
# With sentence-transformers:
# from sentence_transformers import SentenceTransformer
# model = SentenceTransformer('all-MiniLM-L6-v2')
# embeddings = model.encode(sentences)
# node_features = torch.tensor(embeddings) # [N, 384]
Step 2: Edge Construction — Co-occurrence Window
A co-occurrence window connects each sentence to the sentences that follow it
within a window of size w. This captures local narrative flow.
def build_window_edges(N: int, window: int = 2) -> torch.Tensor:
src, dst = [], []
for i in range(N):
for j in range(i+1, min(i+window+1, N)):
src.append(i)
dst.append(j)
src.append(j) # bidirectional
dst.append(i)
return torch.tensor([src, dst], dtype=torch.long) # [2, E]
N = len(sentences)
edge_index = build_window_edges(N, window=2)
print(f"Window edges shape: {edge_index.shape}") # [2, E]
For N=5 and window=2, this produces edges between sentences within 2
positions of each other.
Step 3: Edge Construction — Semantic Similarity
Semantic similarity edges connect sentences with similar embeddings,
regardless of position. Use build_knn_graph from TGraphX:
from tgraphx.graph_builders import build_knn_graph
# k=2: each sentence connects to its 2 most similar sentences
edge_index_semantic = build_knn_graph(node_features, k=2) # [2, E]
print(f"Semantic edges shape: {edge_index_semantic.shape}")
build_knn_graph returns a [2, E] edge index compatible with TGraphX's
Graph constructor. The edges are directed (from each node to its k neighbors).
Step 4: Assemble the Graph
from tgraphx import Graph
from tgraphx.performance import recommended_device
device = recommended_device()
# Choose edge construction strategy
edge_index = edge_index_semantic # or edge_index from window approach
E = edge_index.shape[1]
edge_features = torch.zeros(E, 1) # no edge features for now
# Labels: sentence-level, e.g., which topic cluster
node_labels = torch.randint(0, 3, (N,)) # [N] — 3 topic classes
g = Graph(
node_features=node_features, # [N, d]
edge_index=edge_index, # [2, E]
edge_features=edge_features, # [E, 1]
node_labels=node_labels, # [N]
).to(device)
print(f"Graph: {N} nodes, {E} edges on {device}")
print(f"node_features: {g.node_features.shape}")
print(f"edge_index: {g.edge_index.shape}")
ASCII Text-to-Graph Pipeline Diagram
Text-to-Graph Pipeline
──────────────────────────────────────────────────────────────
Raw text
"Sentence 0. Sentence 1. Sentence 2. Sentence 3. Sentence 4."
↓
Sentence splitting
[s0, s1, s2, s3, s4]
↓
Sentence embedding (encoder)
node_features [N=5, d=128]
┌─────────────────────────────────┐
│ Edge construction │
│ │
│ Window approach: │
│ s0─s1, s0─s2, s1─s2, ... │
│ │
│ OR │
│ │
│ k-NN semantic: │
│ build_knn_graph(feats, k=2) │
└─────────────────────────────────┘
↓
edge_index [2, E]
↓
Graph(node_features, edge_index, ...)
↓
TGraphX Graph object, validated, device-aware
↓
GNN model forward pass
──────────────────────────────────────────────────────────────
Batching Multiple Documents
For a dataset of multiple documents:
from tgraphx import GraphBatch, GraphDataLoader
def text_to_graph(sentences, embeddings, labels):
N = len(sentences)
node_features = embeddings # [N, d]
edge_index = build_knn_graph(node_features, k=2)
E = edge_index.shape[1]
return Graph(
node_features=node_features,
edge_index=edge_index,
edge_features=torch.zeros(E, 1),
node_labels=labels,
)
# List of Graph objects, one per document
graphs = [text_to_graph(sents, embs, labs) for sents, embs, labs in dataset]
batch = GraphBatch.from_graphs(graphs[:32])
batch = batch.to(device)
Limitations and Honest Caveats
Embedding quality matters most: The graph structure is only as good as
the node features. A poorly-trained sentence encoder produces uninformative
node features, and the GNN cannot compensate.
Window size is a hyperparameter: There is no universal correct window
size. For narrative text, window=2 to 3 is a reasonable starting point.
k-NN is not semantic understanding: k-NN connects by vector distance in
embedding space. If the embeddings are not semantically meaningful, k-NN
edges are not semantically meaningful either.
This approach is appropriate for research exploration, not a claim that
text graphs outperform transformer models on standard NLP benchmarks.
Using write_graph_stats for Monitoring
After constructing a text graph, record its structure:
from tgraphx import write_graph_stats
write_graph_stats(g, path="./logs/text_graph_stats.json")
# Records: N nodes, E edges, feature shapes, device
This is particularly useful when the graph construction method (window vs
k-NN, embedding model choice) varies across experiments — the stats file
gives you a quick check that construction produced the expected structure.