GLANCE for Context: Learning When to Leverage LLMs for Node-Aware GNN-LLM Fusion

Donald Loveland, Yao-An Yang & Danai Koutra

Department of Computer Science and Engineering
University of Michigan
Ann Arbor, MI, USA
{dlovelan, ayayang, dkoutra}@umich.edu

ABSTRACT

Learning on text-attributed graphs has motivated the use of Large Language Models (LLMs) for graph learning. However, most fusion strategies are applied uniformly across all nodes and attain only small overall performance gains. We argue this result stems from aggregate metrics that obscure when LLMs provide benefit, inhibiting actionable signals for new strategies. In this work, we reframe LLM–GNN fusion around nodes where GNNs typically falter. We first show that performance can significantly differ between GNNs and LLMs, with each excelling on distinct structural patterns, such as local homophily. To leverage this finding, we propose GLANCE (GNN with LLM Assistance for Neighborand Context-aware Embeddings), a framework that invokes an LLM to refine a GNN’s prediction. GLANCE employs a lightweight router that, given inexpensive per-node signals, decides whether to query the LLM. Since the LLM calls are non-differentiable, the router is trained with an advantage-based objective that compares the utility of querying the LLM against relying solely on the GNN. Across multiple benchmarks, GLANCE achieves the best performance balance across node subgroups, achieving significant gains on heterophilous nodes (up to +13%) while simultaneously achieving top overall performance. Our findings highlight the value of adaptive, node-aware GNN-LLM architectures, where selectively invoking the LLM enables scalable deployment on large graphs without incurring high computational costs.

1 INTRODUCTION

Text is rarely consumed in isolation: scientific papers cite related work (Ciotti et al., 2015; Mccallum et al., 2000; Cohan et al., 2018), users browse descriptions of co-purchased e-commerce items (Jin et al., 2024; McAuley & Yang, 2016; Ezeife & Karlapalepu, 2023), and social media posts reply to one another (Wu et al., 2017; Yang & Leskovec, 2011). These interactions form text-attributed graphs (TAGs), where nodes represent text and edges capture relationships, enabling joint reasoning over content and structure (Yang et al., 2015; Zhao et al., 2023). Historically, TAGs have been processed by feeding shallow text features, such as TF–IDF or static word embeddings (Stephen et al., 2022; Mikolov et al., 2013), into Graph Neural Networks (GNNs). Recently, the success of Large Language Models (LLMs) has motivated hybrid architectures that leverage LLMs for graph learning, combining LLMs’ semantic reasoning with GNNs’ structural learning (Chen et al., 2024a; Wang et al., 2025). However, the majority of hybrid systems typically apply a single fusion strategy across all nodes in a graph, overlooking per-node variations in semantic quality and structural attributes (Wu et al., 2024). This uniform application of LLMs across a graph can waste expensive LLM calls on nodes already well modeled by the GNN, producing poor accuracy-efficiency tradeoffs (Liu et al., 2025).

A key weakness of uniform fusion is its failure to leverage the complementary strengths of GNNs and LLMs. GNNs tend to perform well when nodes have a high degree and their neighbors exhibit homophily (i.e., connected nodes share labels). (McPherson et al., 2001; Yan et al., 2022). Yet, these properties do not typically hold in real-world TAGs (Ma et al., 2020; Loveland et al., 2024; Zhou et al., 2020). While advanced GNN designs have attempted to address these concerns (Zhu et al., 2020; Abu-El-Haija et al., 2019; Veliˇckovi´c et al., 2018), recent evidence shows they remain insufficient for handling these challenging structures (Loveland & Koutra, 2025; Mao et al., 2023; Du et al., 2022). On the other hand, LLMs exhibit strong generalization in low-shot settings, making them well suited for the challenging nodes that degrade GNNs (Chen et al., 2024a;

Peng et al., 2024). However, leveraging LLMs for graphs often leads to the distortion of structural relationships due to serializing the graph into text (Liu et al., 2023; Firooz et al., 2025; Wang et al., 2025). This limitation is particularly detrimental for graphs governed by simpler structural signals where the LLM may add unnecessary complexity and even degrade performance relative to GNNs (Liu et al., 2025; Wang et al., 2025).

While incremental performance gains with high computational costs make LLMs seem ill-suited for graph learning, we argue this stems not from inherent deficiencies, but how LLMs are used. Our hypothesis is that gains in regions difficult for GNNs are offset by losses elsewhere. Thus, we focus on utilizing LLMs only where GNNs struggle, prioritizing structurally difficult nodes whose errors are often masked in aggregate metrics. This emphasis promotes equity across the graph, countering harmful GNN inductive biases (Wang et al., 2022; Agarwal et al., 2021). This leads us to our core research question: How, and for which nodes, should we leverage LLMs to complement and bolster GNNs?

To answer this question, we first identify signals that inform when GNNs vs. LLMs succeed. Since LLM routing has only recently been studied, we systematically assess existing methods (Qiao et al., 2025; Jaiswal et al., 2024; Chen et al., 2024b), finding highly variable performance, sometimes worse than random routing. Through a node-level analysis on local homophily and relative degree, two properties known to impact GNNs (Yan et al., 2022; Subramonian et al., 2024), we find GNNs excel in high-homophily, well-connected regions, while LLMs dominate as homophily or degree decreases. Notably, local homophily emerges as a strong predictor of model advantage, offering a principled prior for routing.

Motivated by this, we propose GLANCE (GNN with LLM Assistance for Neighborand Context-aware Embeddings), a fusion strategy that preserves GNN efficiency on easy nodes while selectively leveraging LLMs on hard ones. GLANCE first encodes nodes with a pre-trained GNN, then passes lightweight routing features to a cost-aware policy that decides whether to query the LLM – figuratively “glancing” at the LLM for additional context. For routed nodes, LLM embeddings are fused with GNN embeddings via a small refiner head. To train the non-differentiable router, we introduce an advantage-based strategy that rewards beneficial queries to the LLM. We find that GLANCE is able to consistently outperform previous benchmarks, producing more robust predictions across the spectrum of homophily levels. This ability to achieve higher performance while reducing LLM usage, and thus computational cost, highlights the value of node-aware GNN-LLM fusion when scaling to large, diverse TAGs. Our contributions are below:

• Current Limitations: Which Nodes to Route. We provide a systematic look at LLM routing and show that current heuristics are brittle. By stratifying nodes by different properties, we reveal the complementary performance of GNNs and LLMs over local homophily and identify for which nodes LLMs are beneficial.
• New GNN-LLM Method: How to Select Nodes. We introduce GLANCE, a cost-aware framework that learns when to query the LLM, minimizing unnecessary LLM calls and preserving scalability.
• Comprehensive Empirical Analysis. Across four diverse TAG datasets, GLANCE consistently outperforms state-of-the-art GNNs and GNN-LLM hybrids, yielding robust predictions across homophily levels with gains of up to +13.0% on heterophilous pockets and overall performance gains of up to +0.9%.

2 RELATED WORK

We briefly discuss the most relevant work here, and provide more details in Appendix A.

Static GNN-LLM Fusion. Prior work largely follows two paradigms: LLMs-as-Enhancers and LLMs-as-Predictors (Chen et al., 2024a; Li et al., 2024). Enhancer methods enrich node features with text embeddings or external knowledge (Wu et al., 2024; He et al., 2024), while predictor methods replace the GNN with the LLM, directly classifying serialized graph inputs (Wu et al., 2024; Wang et al., 2025). LLM-as-Predictor methods can overcome GNN biases, but struggle with topology encoding and prompt length (Firooz et al., 2025). Other fusion strategies draw from Mixture-of-Experts (MoE) (Cai et al., 2025; Wang et al., 2023), but have not explored mixing GNNs and LLMs. Recent work has considered LLMs to route among GNNs (Jiang & Luo, 2025), yet this still confines performance to GNN biases. In contrast, our approach adopts a selective paradigm, invoking the LLM where it is expected to help, combining the strengths of both models.

Adaptive GNN-LLM Fusion. A challenge for LLM–GNN models is inference cost, especially when LLMs are used in the forward pass for all nodes (Wu et al., 2024). While cost reduction via sampling or distillation helps (Fang et al., 2023; Chen et al., 2024a), scalability remains limited by universal LLM calls. Only a few recent works explore adaptive GNN-LLM fusion, where the LLM is invoked selectively. E-LLaGNN

routes nodes with fixed heuristics (e.g., degree, centrality) (Jaiswal et al., 2024), but requires manual tuning. LOGIN uses GNN uncertainty to rewire difficult nodes (Qiao et al., 2025), though this can erase useful heterophilous links (Luan et al., 2021). LLM-GNN uses clustering density as a proxy for difficulty (Chen et al., 2024b). Our approach differs by: (1) systematically identifying structural properties predictive of LLM benefit, learning to route without preset thresholds; (2) preserving graph structure rather than editing it; and (3) directly training a router under non-differentiable queries, rather than altering data to create influence.

3 PRELIMINARIES

A TAG is defined as , where is the set of nodes, is the set of edges, is the text associated with each node , and is the set of node labels. The task is to learn a model that predicts node labels from both structure and text. Next we outline the strategies to parameterize and learn .

for a node at layer . Additionally, (a feature vector from ), denotes the neighbors of node , and AGGREGATE and UPDATE define the GNN’s operations. In practice, AGGREGATE is a permutation-invariant function (e.g. sum or mean) and UPDATE is a multi-layer perceptron (MLP). After layers, the prediction is derived from an MLP head. Despite their success, GNNs can struggle on certain structural patterns, such as low degree and heterophily (Loveland et al., 2024; Mao et al., 2023; Yan et al., 2022), motivating architectures with higher-order aggregation, residual connections, and adaptive message passing (Chen et al., 2020; Zhu et al., 2020). We leverage GNNs that adopt these designs later in our analysis.

Large Language Models. Let denote a pre-trained language model. Given a node with text , and optionally neighbor attributes , the LLM can either (i) generate an embedding from a prompt , optionally including neighborhood context, or (ii) directly predict the label . Embedding LLMs allow the output to be easily combined with a GNN’s output or other downstream classifiers, but require a prediction head. In contrast, direct prediction avoids this head but risks hallucinated labels and often still needs fine-tuning. In this work, we adopt the embedding setting, which supports seamless integration with GNN representations.

4 WHICH NODES TO ROUTE: CURRENT LIMITATIONS & OPPORTUNITIES

We begin by examining which nodes should be routed, reviewing prior routing strategies and highlighting their limitations. Then, building on insights from the GNN literature, particularly the challenges with GNNs applied to low-degree nodes and heterophilous neighborhoods (Yan et al., 2022; Tang et al., 2020), we show that LLMs can complement GNNs under these conditions. Ultimately, we find that homophily emerges as a strong indicator for LLM benefit, offering a promising opportunity to improve graph learning.

4.1 LIMITATIONS OF CURRENT ROUTING STRATEGIES

Based on previous work, we analyze strategies to route nodes to an LLM. We consider node degree (Jaiswal et al., 2024), clustering density (C-density) (Chen et al., 2024b), and uncertainty from dropout (Qiao et al., 2025). For each strategy, we route the top-k% of nodes to a fine-tuned LLM under three criteria: (i) low degree, (ii) low density, and (iii) high uncertainty.

Experimental Setup. We evaluate on Cora (Mccallum et al., 2000), Pubmed (Sen et al., 2008a), and Arxiv23 (He et al., 2024) with processing details in Section D. We train two backbones, GCN and GCNII, using the dataset’s original features (denoted as “Std.”) and enhanced features (denoted as “Enh.”) generated via Qwen3-8B. GCN serves as a traditional baseline, while GCNII introduces new designs to better handle over-smoothing and other challenging graph properties. Together, they provide a contrast between a simple message-passing framework and a stronger state-of-the-art backbone. For LLM-as-predictor, we fine-tune Qwen3-8B with an MLP head for node classification. LLM prompting and training for both LLM-as-Enhancer and LLM-as-Predictor are given in Section B. Training details and hyperparameters for the LLMs and GNNs are provided in Section C. For the routing strategies in Table 1, we freeze the GNN and LLMs and route using the metric.

We assess heuristics with a net correction score (NCS), capturing the benefit of routing. For a routed set , let be the nodes wrong under the GNN but became correct after routing to the LLM, and be the set of nodes were correct but became wrong. We define where NCS=1 means the LLM fixes all routed nodes and −1 means the LLM harmed every routed node.

Findings. Table 1 highlights the difficulty to identify a single routing signal that generalizes across datasets. On Pubmed and Arxiv23, uncertainty appears promising, achieving the best NCS in every setting. Yet on Cora, the same strategy consistently produces negative NCS, often performing worse than a random router. Other heuristics such as degree and C-density similarly fluctuate in effectiveness, with no strategy performing robustly across all datasets. Together, these results underscore the limitations of static heuristics: while they can succeed in isolated cases, their performance is dataset-dependent. This motivates the need for a more principled and transferable routing signal, rather than manually chosen rules.

4.2 STRUCTURAL OPPORTUNITIES FOR ROUTING

Given that heuristic quality can be dataset-dependent, we aim to find a routing signal that aligns with the strengths of GNNs and LLMs. We first analyze metrics known to degrade GNN performance and then utilize these to route, showing that homophily is a strong indicator for when LLMs improve performance.

4.2.1 ON THE COMPLEMENTARY CAPABILITIES OF LLMS AND GNNS

To characterize the unique benefits of LLMs and GNNs, we take the models from Table 1 and perform a stratified analysis using relative degree and local homophily . When , tends to have higher degree than its neighbors, and lower degree otherwise. Low indicates is heterophilous. Mathematically,

Both properties are known to influence GNNs (Yan et al., 2022; Subramonian et al., 2024), but have received less attention for LLM-graph reasoning. We compare models across these properties to understand how each paradigm responds.

Figure 1: Stratified performance for (top) and ¯ (bottom). Bars denote property distributions (right y-axis). Enhanced GNNs can benefit heterophilous & low degree nodes, but LLMs offer further improvements.

Findings. In Figure 1, we study performance for groups of nodes stratified by and ¯. We find that LLM models tend to produce significantly higher performance over heterophilous and low-degree nodes, e.g., achieving up to a 20.4% performance increase compared to the next best model, GCNII with LLM enhancement on Cora. Additional results given for Pubmed in Section E.4 with similar trends. In Figure 4 (Section E.5), we further find that homophily and degree can interplay with one another, where performance differences can reach upwards of 30.1% between subpopulations stratified by both degree and local homophily. With these findings, we next study if homophily can be useful for routing.

4.2.2 ROUTING WITH LOCAL HOMOPHILY

Unlike degree, which can be derived directly from the graph structure, homophily depends on class labels that are not accessible during training. This makes it challenging to directly exploit the performance gains LLMs provide on heterophilous nodes (Figure 1). Motivated by prior work on classifying edges as homophilous or heterophilous (Du et al., 2022; Wu et al., 2024), we extend this idea to estimate a node-level homophily score. Concretely, we train an MLP to predict node labels, , and use these predictions to compute an estimated local homophily,

Findings. Table 2 shows that homophily is a useful routing signal. First, we find that true local homophily, , attains the highest NCS in most settings, establishing an upper bound for structure-aware routing. More importantly, our label-free homophily estimate closely tracks and typically matches or surpasses other static heuristics. The average rankings in Table 2 corroborates this finding, where has the best mean rank for NCS score. Moreover, when is excluded from the ranking, our estimated homophily achieves the best average rank. Overall, homophily reliably identifies nodes that benefit from LLM assistance, and our label-free homophily proxy is a practical and effective prior for routing.

5 HOW TO ROUTE: ADAPTIVELY FUSING LLMS AND GNNS WITH GLANCE

To address the limitations of existing routing strategies, we propose GLANCE, a framework that adaptively fuses GNNs and LLMs. Rather than relying on handcrafted rules, GLANCE employs a lightweight router trained to decide, on a per-node basis, whether to invoke the LLM. This design ensures that the LLM is used only when it provides improvement to justify its cost. GLANCE is comprised of three components: (i) frozen GNN and LLM encoders, (ii) a trainable router using cheap features for routing, and (iii) a combiner that fuses structure and text embeddings into a final prediction.

5.1 COMPONENTS OF GLANCE

We now detail the components of GLANCE, as depicted in Figure 2. We first outline how nodes are chosen for routing. Then, we specify how the LLM generates new embeddings for routed nodes. Finally, we define the refinement process that merges the GNN and LLM information. As this is a non-differentiable pipeline, we include details on how the router is trained to encourage effective usage of the LLM.

5.1.1 STEP 1: GENERATING AND PROCESSING ROUTING FEATURES.

Routing Features. We begin by using a pre-trained GNN to produce embedding from the -hop neighborhood of v. This embedding acts as the first signal to ensure the router can leverage the structural information of the neighborhood. We also use the GNN to derive uncertainty estimations on the node, following a dropout strategy (Qiao et al., 2025). This uncertainty acts as a proxy signal for difficulty, allowing the router to take advantage of the relationships found in Table 1. Building on the opportunity to leverage homophily, we utilize to attain a local homophily estimate. However, to increase expressivity, we compute the probability distribution over the classes, , and estimate a soft local homophily as:

Figure 2: GLANCE Overview. Step 1: GLANCE generates routing features to derive a decision. Step 2: A routed node’s text is fed into the LLM to generate embeddings. Step 3: A routed node’s GNN & LLM embeddings are used to refine predictions. For nodes not routed, the GNN prediction is used. Only the router and refiner MLP are trained.

This soft estimate provides the router with a measure of neighborhood alignment. While Table 2 shows that cannot definitively route on its own, we utilize this signal as a prior to bias the router toward heterophilous nodes, which we expect to be refined during training. While uncertainty was originally motivated to capture heterophily Qiao et al. (2025), we find this correlation to be weak (seen in Table 9), and use both metrics given they provide different, yet informative signals. Finally, we include the original features and degree, enabling the routing of nodes with noisy features or insufficient neighborhood context.

Node Router. We define a router that takes as input the routing features above, denoted as a vector , and outputs a probability of routing , for a node v, as , where larger values of indicate that the LLM should be queried for node v, while low values of indicates that the GNN prediction is sufficient for node v. Rather than applying an absolute threshold, GLANCE uses a top-k strategy: for each mini-batch, the k nodes with the highest scores are routed to the LLM. This ensures a fixed query budget per batch and avoids the need to globally calibrate the router’s probabilities.

5.1.2 STEP 2: PRE-TRAINED LLM TO PROCESS ROUTED NEIGHBORHOODS

For the routed node set , the LLM encoder is used to generate embeddings from serialized neighborhood prompts , examples prompts given in Section B. Rather than producing a single embedding as in previous work (Wang et al., 2025), we generate embeddings at multiple structural levels: (1) the ego text , (2) the ego text with a sampled set of 1-hop neighbors , (3) the ego text with a sampled set of 2-hop neighbors . Each level is serialized into separate prompts, encoded into embeddings , and concatenated to form the final embedding:

This encoding preserves both ego and neighbor information, while ensuring the prompt length remains manageable. Additionally, this design aligns with the aggregation typically seen in advanced GNNs (Zhu et al., 2020; Yan et al., 2022). Using LLM embeddings also offers computational advantages, as it avoids the costly generation step required by prior approaches (Qiao et al., 2025; Wu et al., 2024).

5.1.3 STEP 3: REFINE THE GNN PREDICTIONS USING LLM EMBEDDINGS.

For nodes not routed, we retain their original GNN predictions by feeding the GNN-based node embeddings into the original MLP head used during GNN training. For nodes that are routed, we want to utilize both the GNN and LLM embeddings to capture structure and contextual information. Thus, we define a refiner MLP C that integrates the GNN and LLM embedding as:

where is the probability distribution over the classes for a node v. This modular design is agnostic to a specific GNN or LLM backbone, enabling flexibility depending on the dataset or computational budget.

5.2 TRAINING OBJECTIVES

Because routing decisions are discrete and require prompt construction, routing weights cannot be directly learned via backpropagation. Below, we detail how we translate the routing signal into weight updates.

Counterfactual Comparison and Rewards. When routing to the LLM, we measure the value of invoking it by computing a counterfactual prediction using the GNN and measuring the difference in losses produced by the two computations. Specifically, with and the GNN’s pre-trained prediction head :

where is the loss when routing with the LLM, and is the counterfactual loss from the GNN. Note, the counterfactual is only computed when nodes are routed to the LLM during training, i.e. we do not compute an LLM-based counterfactual for nodes not routed. We then quantify the benefit of routing (or not routing) to the LLM through the rewards:

The first term captures the decrease in loss provided by querying the LLM and the cost of invoking the LLM (hyperparameter ≥0). Larger imposes stricter penalties, discouraging the use of the LLM where it doesn’t provide benefit. A positive with LLM routing indicates that calling the LLM reduced prediction loss enough to offset its cost. When the LLM is not used, the reward is based on the GNN loss.

Training Objective. The router is optimized with a policy gradient-inspired loss, treating routing as a contextual bandit problem and encouraging alignment with counterfactual advantage:

While this objective is inspired by REINFORCE (Sutton et al., 1999), we employ deterministic top-k selection during training to ensure stable use of the limited LLM query budget. The final objective jointly trains C and through:

We use to denote the prediction loss coming from either the LLM or GNN computation paths. Then, the first term of optimizes predictive performance, while the second enforces cost-aware routing. For efficiency, both the GNN and the LLM are kept frozen during training, training only and .

6 EMPIRICAL ANALYSIS

We now empirically study GLANCE, demonstrating how selectively using LLMs can improve performance. We first analyze GLANCE’s performance, showing that it achieves significantly more balanced performance compared to the baselines. Then, we provide a series of supplemental analyses to understand what GLANCE learns during training. Finally, we extend GLANCE to larger TAGs to demonstrate its scalability.

6.1 EXPERIMENTAL SETUP

Datasets. We first evaluate across Cora (Mccallum et al., 2000), Pubmed (Sen et al., 2008b), Arxiv23 (He et al., 2024), studying overall and stratified performance, as well as performing numerous sensitivity and ablation analyses. Then, to demonstrate GLANCE’s scalability, we evaluate on two large-scale datasets, Arxiv-Year (Lim et al., 2021) and OGB-Products, with details provided in Section D. For each dataset, we follow the same training set up as outlined in the Section 4.1.

Baselines & Training. We compare GLANCE to a series of state-of-the-art baselines for scalable LLMGNN fusion. We start by considering standard GNN backbones, including GCN (Kipf & Welling, 2017), GraphSAGE (Hamilton et al., 2017), and GCNII (Chen et al., 2020). We study their performance over three settings, original features, LLM-enhanced features, and LOGIN-filtered graphs. As GLANCE’s core design is to bolster challenging heterophilous nodes, we also include high-performing heterophilous GNNs, including FAGCN, GGCN, and GBK-GNN. To imbue them with LLM knowledge, we utilize LLMenhanced features. We do not apply LOGIN on these models as they are designed to utilize heterophily, whereas LOGIN’s design intends to remove it. Across all baselines, we use identical data splits, training protocols, and text encoders. Training details and hyperparameters are provided in Section C. Unless otherwise stated, we route the top 12 nodes per batch of 32 nodes for GLANCE for reported metrics. For OGB-Products, we evaluate against the top 3 model combinations found in our initial study.

6.2 CAN GLANCE ACHIEVE THE BENEFITS OF LLMS AND GNNS IN ONE MODEL?

As shown in Figure 1, selectively using the LLM on challenging nodes can enhance GNN performance. To evaluate GLANCE’s capabilities, we report both overall accuracy (Table 4) and stratified results (Table 3). GLANCE attains the best performance on all three datasets, achieving 89.5% on Cora, 92.6% on Pubmed, and 82.1% on Arxiv23, leading to an average gain of +0.5% over the next best model. Notably, GLANCE requires significantly fewer LLM queries to attain similar performance gains found in previous work (Wang et al., 2025; Chen et al., 2024a). GLANCE also consistently outperforms LLM-enhanced and LOGIN baselines, often by several percent, as well as GNNs designed for heterophily. From a stratified perspective, the largest improvements emerge on heterophilous nodes, where GLANCE delivers +13% on Cora and +0.5% on Pubmed. Importantly, when averaging across bins, GLANCE achieves the strongest overall balance with an average rank of 2.4, compared to 4.7 for the next best. Together, these findings demonstrate that GLANCE’s routing reliably leverages the LLM where it is most beneficial while preserving the strengths of the GNN.

Properties of Routed Nodes. In Figure 3, we analyze the local homophily of routed nodes, splitting them into benefited (GNN wrong, LLM right) and non-benefited sets. Across datasets, we observe substantial mass at low homophily, showing that GLANCE preferentially routes heterophilous nodes. Crucially, the benefit distribution is skewed toward low homophily, indicating that routed heterophilous nodes deliver performance gains. This supports our hypothesis that heterophily captures GNN failure modes where LLMs are most valuable, and shows that these nodes drive GLANCE’s improvement. We also highlight that the median beneficial homophily value is not identical across datasets, e.g., on Arxiv23 the largest gains occur around . Thus, a single homophily threshold that only routes heterophilous nodes would be ineffective for maximizing performance, and additional context is needed.

Figure 3: Local homophily for routed nodes, split by benefit. Blue lines denote median.

Sensitivity to . In Figure 6 (Section F.1), we examine performance under varying routing budgets. For a batch size of 32, we evaluate . We find that increasing tends to lead to steady gains on heterophilous nodes, while leaving performance on the homophilous nodes mostly unchanged. On Pubmed and Arxiv23, accuracy on improves by +3.4% for , and by another +3.0% for . On Cora, performance initially dips for , but then significantly improves by +12.3% when increasing to . For , GLANCE’s accuracy drops by a marginal −0.06% on average across datasets, underscoring GLANCE’s stability in this regime. Importantly, we also observe that routing too many nodes can harm accuracy (as seen in Table 10 for full-batch routing), as easy nodes well modeled by the GNN can become misclassified when the LLM is unnecessarily used. Together, these findings demonstrate that GLANCE routes efficiently, mitigating both the performance degradation and computational overhead of indiscriminate LLM usage.

6.3 ROUTER: LEARNED PROPERTIES AND ROBUSTNESS

Routing Feature Ablation. To assess the value of the routing features, we disable each feature one at a time and train new GLANCE models, measuring − (Figures in Section F.3). Averaged over all features, accuracy drops on each dataset, with changes of −0.38% on Cora, −1.07% on Pubmed, and −0.65% on Arxiv23. More importantly, when ablating the homophily feature, we find significant drops when <0.5, with changes of −6.5% for Cora, −6.3% for Pubmed, and −2.0% for Arxiv23. We also observe subtle losses when >0.5 as the homophily signal is removed, where Cora drops by −0.34%, Pubmed drops by −0.15%, and Arxiv23 drops by −0.03%. These drops indicate that homophily helps the router avoid misrouting the nodes easy for GNNs. Together, these ablations show that GLANCE achieves peak performance when all features are available, with the largest gains on heterophilous nodes.

6.4 LARGE-SCALE LEARNING WITH GLANCE

Datasets. GLANCE’s selective utilization of the LLM enables the processing of large TAGs by only applying the LLM to nodes that benefit. Now that we have demonstrated the effectiveness of GLANCE, we study the scalability by applying GLANCE to two large benchmark datasets, Arxiv-Year (Lim et al., 2021) and OGB-Products (Hu et al., 2020). Details for these datasets can be found in Section D. Notably, Arxiv-Year and OGB-Products are multiple orders of magnitude larger than the datasets typically studied in GNN-LLM papers (Qiao et al., 2025; Wu et al., 2024; Wang et al., 2025). Furthermore, to motivate GLANCE’s scalability, we show in Figure 9 that the routing and refinement components of GLANCE create very little overhead, enabling it to handle larger datasets when querying less nodes with the LLM.

Results. In Table 5, we provide per-bin and overall accuracy for the top 3 average rank models from Table 3 and compare them to GLANCE. Given these datasets are significantly larger than the previous datasets, we utilize a query rate of ( =1 with batch size 64). We find that GLANCE outperforms

the best baselines while achieving strong performance across the spectrum of homophily levels. These results demonstrate that selective LLM queries, even at very low query rates, can enable gains and supplement GNNs where they struggle. Given the results seen above with regards to sensitivity to , we also expect this performance can become even better with a larger compute budget. Moreover, for applications that necessitate better performance on heterophilous pockets of nodes, GLANCE is able to offer key benefits, regardless of dataset size, that are otherwise inaccessible with standard GNN architectures.

7 CONCLUSION

In this work we focused on how LLMs can be selectively utilized to bolster GNNs, an understudied area in the LLM-GNN literature. Through a detailed analysis of structural properties, we showed that homophily provides a reliable signal for routing, capturing where GNNs tend to fail and LLMs excel. To operationalize this finding, we proposed GLANCE, a cost-aware fusion strategy that learns when to route, calling the LLM on difficult nodes. Empirical results across multiple benchmarks demonstrate that GLANCE achieves well-balanced performance while selectively invoking the LLM only when useful. Our findings argue for structure-aware routing as a foundation for future work on efficient GNN-LLM integration.

8 ACKNOWLEDGMENTS

The material in this work is supported by the National Science Foundation under IIS 2504090, IIS 2212143, and CAREER Grant No.IIS 1845491. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or other funding parties.

REFERENCES

Sami Abu-El-Haija, Bryan Perozzi, Amol Kapoor, Hrayr Harutyunyan, Nazanin Alipourfard, Kristina Lerman, Greg Ver Steeg, and Aram Galstyan. Mixhop: Higher-order graph convolution architectures via sparsified neighborhood mixing. In International Conference on Machine Learning (ICML), 2019.

Chirag Agarwal, Himabindu Lakkaraju, and Marinka Zitnik. Towards a unified framework for fair and stable graph representation learning, 2021. URL https://arxiv.org/abs/2102.13186.

Weilin Cai, Juyong Jiang, Fan Wang, Jing Tang, Sunghun Kim, and Jiayi Huang. A survey on mixture of experts in large language models. IEEE Transactions on Knowledge and Data Engineering, pp. 1–20, 2025. ISSN 2326-3865. doi: 10.1109/tkde.2025.3554028. URL http://dx.doi.org/10.1109/ TKDE.2025.3554028.

Ming Chen, Zhewei Wei, Zengfeng Huang, Bolin Ding, and Yaliang Li. Simple and deep graph convolutional networks. In International Conference on Machine Learning, 2020.

Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, and Jiliang Tang. Exploring the potential of large language models (llms) in learning on graphs, 2024a. URL https://arxiv.org/abs/2307.03393.

Zhikai Chen, Haitao Mao, Hongzhi Wen, Haoyu Han, Wei Jin, Haiyang Zhang, Hui Liu, and Jiliang Tang. Label-free node classification on graphs with large language models (llms), 2024b. URL https://arxiv.org/abs/2310.04668.

Valerio Ciotti, Moreno Bonaventura, Vincenzo Nicosia, Pietro Panzarasa, and Vito Latora. Homophily and missing links in citation networks. EPJ Data Science, 5, 11 2015.

Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. A discourse-aware attention model for abstractive summarization of long documents. In Marilyn Walker, Heng Ji, and Amanda Stent (eds.), Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp. 615–621, New Orleans, Louisiana, June 2018. Association for Computational Linguistics. doi: 10.18653/v1/N18-2097. URL https://aclanthology.org/N18-2097/.

Lun Du, Xiaozhou Shi, Qiang Fu, Xiaojun Ma, Hengyu Liu, Shi Han, and Dongmei Zhang. Gbk-gnn: Gated bi-kernel graph neural networks for modeling both homophily and heterophily. In Proceedings of the ACM Web Conference 2022, pp. 1550–1558, 2022.

Christie I. Ezeife and Hemni Karlapalepu. A survey of sequential pattern based e-commerce recommendation systems. Algorithms, 16(10), 2023. ISSN 1999-4893. doi: 10.3390/a16100467. URL https://www.mdpi.com/1999-4893/16/10/467.

Taoran Fang, Yunchao Mercer Zhang, Yang Yang, Chunping Wang, and Lei CHEN. Universal prompt tuning for graph neural networks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=0LmWBhIYLi.

Hamed Firooz, Maziar Sanjabi, Wenlong Jiang, and Xiaoling Zhai. Lost-in-distance: Impact of contextual proximity on llm performance in graph tasks, 2025. URL https://arxiv.org/abs/2410. 01985.

Will Hamilton, Zhitao Ying, and Jure Leskovec. Inductive representation learning on large graphs. In Advances in Neural Information Processing Systems, 2017.

Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun, and Bryan Hooi. Harnessing explanations: Llm-to-lm interpreter for enhanced text-attributed graph representation learning, 2024. URL https://arxiv.org/abs/2305.19523.

Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, and Jure Leskovec. Open graph benchmark: datasets for machine learning on graphs. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.

Ajay Jaiswal, Nurendra Choudhary, Ravinarayana Adkathimar, Muthu P. Alagappan, Gaurush Hiranandani, Ying Ding, Zhangyang Wang, Edward W Huang, and Karthik Subbian. All against some: Efficient integration of large language models for message passing in graph neural networks, 2024. URL https://arxiv.org/abs/2407.14996.

Dapeng Jiang and Xiao Luo. Marrying LLMs with dynamic forecasting: A graph mixture-of-expert perspective. In Luis Chiruzzo, Alan Ritter, and Lu Wang (eds.), Findings of the Association for Computational Linguistics: NAACL 2025, pp. 396–410, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-195-7. doi: 10.18653/v1/2025.findings-naacl.24. URL https://aclanthology.org/2025.findings-naacl.24/.

Wei Jin, Haitao Mao, Zheng Li, Haoming Jiang, Chen Luo, Hongzhi Wen, Haoyu Han, Hanqing Lu, Zhengyang Wang, Ruirui Li, et al. Amazon-m2: A multilingual multi-locale shopping session dataset for recommendation and text generation. Advances in Neural Information Processing Systems, 36, 2024.

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017.

Yuhan Li, Zhixun Li, Peisong Wang, Jia Li, Xiangguo Sun, Hong Cheng, and Jeffrey Xu Yu. A survey of graph meets large language model: Progress and future directions, 2024. URL https: //arxiv.org/abs/2311.12399.

Derek Lim, Felix Hohne, Xiuyu Li, Sijia Linda Huang, Vaishnavi Gupta, Omkar Bhalerao, and Ser Nam Lim. Large scale learning on non-homophilous graphs: New benchmarks and strong simple methods. In NeurIPS, volume 34. Curran Associates, Inc., 2021.

Jiacheng Lin, Kun Qian, Haoyu Han, Nurendra Choudhary, Tianxin Wei, Zhongruo Wang, Sahika Genc, Edward W Huang, Sheng Wang, Karthik Subbian, Danai Koutra, and Jimeng Sun. Gt2vec: Large language models as multi-modal encoders for text and graph-structured data, 2025. URL https://arxiv.org/abs/2410.11235.

Jiajin Liu, Dongzhe Fan, Jiacheng Shen, Chuanhao Ji, Daochen Zha, and Qiaoyu Tan. Graph-mllm: Harnessing multimodal large language models for multimodal graph learning, 2025. URL https: //arxiv.org/abs/2506.10282.

Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts, 2023. URL https: //arxiv.org/abs/2307.03172.

Donald Loveland and Danai Koutra. Unveiling the impact of local homophily on gnn fairness: In-depth analysis and new benchmarks. SDM, 2025.

Donald Loveland, Jiong Zhu, Mark Heimann, Benjamin Fish, Michael T Schaub, and Danai Koutra. On performance discrepancies across local homophily levels in graph neural networks. In LoG, volume 231. PMLR, 27–30 Nov 2024.

Sitao Luan, Chenqing Hua, Qincheng Lu, Jiaqi Zhu, Mingde Zhao, Shuyuan Zhang, Xiao-Wen Chang, and Doina Precup. Is heterophily a real nightmare for graph neural networks to do node classification?, 2021. URL https://arxiv.org/abs/2109.05641.

Jiaqi Ma, Shuangrui Ding, and Qiaozhu Mei. Towards more practical adversarial attacks on graph neural networks. Advances in neural information processing systems, 2020.

Haitao Mao, Zhikai Chen, Wei Jin, Haoyu Han, Yao Ma, Tong Zhao, Neil Shah, and Jiliang Tang. Demystifying structural disparity in graph neural networks: Can one size fit all? In NeurIPS, volume 36, 2023.

Julian McAuley and Alex Yang. Addressing complex and subjective product-related queries with customer reviews. In Proceedings of the 25th International Conference on World Wide Web, WWW ’16, pp. 625–635, Republic and Canton of Geneva, CHE, 2016. International World Wide Web Conferences Steering Committee. ISBN 9781450341431. doi: 10.1145/2872427.2883044. URL https://doi. org/10.1145/2872427.2883044.

Andrew Mccallum, Kamal Nigam, Jason Rennie, and Kristie Seymore. Automating the construction of internet portals with machine learning. Information Retrieval, 3(2), 11 2000.

Miller McPherson, Lynn Smith-Lovin, and James M Cook. Birds of a feather: Homophily in social networks. Annual review of sociology, 2001.

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. Efficient estimation of word representations in vector space, 2013. URL https://arxiv.org/abs/1301.3781.

Jingyu Peng, Qi Liu, Linan Yue, Zaixi Zhang, Kai Zhang, and Yunhao Sha. Towards few-shot selfexplaining graph neural networks, 2024. URL https://arxiv.org/abs/2408.07340.

Yiran Qiao, Xiang Ao, Yang Liu, Jiarong Xu, Xiaoqian Sun, and Qing He. Login: A large language model consulted graph neural network training framework. In Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining, WSDM ’25, pp. 232–241, New York, NY, USA, 2025. Association for Computing Machinery. ISBN 9798400713293. doi: 10.1145/3701551.3703488. URL https://doi.org/10.1145/3701551.3703488.

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI Magazine, 29(3):93, Sep. 2008a. doi: 10.1609/aimag. v29i3.2157. URL https://ojs.aaai.org/aimagazine/index.php/aimagazine/ article/view/2157.

Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI Magazine, 29(3):93, Sep. 2008b. doi: 10.1609/aimag. v29i3.2157. URL https://ojs.aaai.org/aimagazine/index.php/aimagazine/ article/view/2157.

Akuma Stephen, Tyosar Lubem, and Isaac Adom. Comparing bag of words and tf-idf with different models for hate speech detection from live tweets. International Journal of Information Technology, 14, 09 2022. doi: 10.1007/s41870-022-01096-4.

Arjun Subramonian, Jian Kang, and Yizhou Sun. Theoretical and empirical insights into the origins of degree bias in graph neural networks. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL https://openreview.net/forum?id=1mAaewThcz.

Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In S. Solla, T. Leen, and . M¨uller (eds.), Advances in Neural Information Processing Systems, volume 12. MIT Press, 1999. URL https://proceedings.neurips.cc/paper_files/paper/1999/file/ 464d828b85b0bed98e80ade0a5c43b0f-Paper.pdf.

Xianfeng Tang, Huaxiu Yao, Yiwei Sun, Yiqi Wang, Jiliang Tang, Charu C. Aggarwal, Prasenjit Mitra, and Suhang Wang. Investigating and mitigating degree-related biases in graph convolutional networks. Proceedings of the 29th ACM International Conference on Information and Knowledge Management, 2020.

Petar Veliˇckovi´c, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Li`o, and Yoshua Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.

Haotao Wang, Ziyu Jiang, Yuning You, Yan Han, Gaowen Liu, Jayanth Srinivasa, Ramana Kompella, and Zhangyang ”Atlas” Wang. Graph mixture of experts: Learning on large-scale graphs with explicit diversity modeling. In A. Oh, T. Naumann, A. Globerson, . Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 50825–50837. Curran Associates, Inc., 2023. URL https://proceedings.neurips.cc/paper_files/paper/ 2023/file/9f4064d145bad5e361206c3303bda7b8-Paper-Conference.pdf.

Yu Wang, Yuying Zhao, Yushun Dong, Huiyuan Chen, Jundong Li, and Tyler Derr. Improving fairness in graph neural networks via mitigating sensitive attribute leakage. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’22, pp. 1938–1948. ACM, August 2022. doi: 10.1145/3534678.3539404. URL http://dx.doi.org/10.1145/3534678. 3539404.

Yuxiang Wang, Xinnan Dai, Wenqi Fan, and Yao Ma. Exploring graph tasks with pure llms: A comprehensive benchmark and investigation, 2025. URL https://arxiv.org/abs/2502.18771.

Bo Wu, Wen-Huang Cheng, Yongdong Zhang, Qiushi Huang, Jintao Li, and Tao Mei. Sequential prediction of social media popularity with deep temporal context networks. arXiv preprint arXiv:1712.04443, 2017.

Yuxia Wu, Shujie Li, Yuan Fang, and Chuan Shi. Exploring the potential of large language models for heterophilic graphs. arXiv preprint arXiv:2408.14134, 2024.

Yujun Yan, Milad Hashemi, Kevin Swersky, Yaoqing Yang, and Danai Koutra. Two sides of the same coin: Heterophily and oversmoothing in graph convolutional neural networks. In 2022 IEEE International Conference on Data Mining (ICDM), pp. 1287–1292, 2022.

Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. Network representation learning with rich text information. In IJCAI, pp. 2111–2117, 2015.

Jaewon Yang and Jure Leskovec. Patterns of temporal variation in online media. In Proceedings of the Fourth ACM International Conference on Web Search and Data Mining, WSDM ’11, pp. 177–186, New York, NY, USA, 2011. Association for Computing Machinery. ISBN 9781450304931. doi: 10.1145/1935826.1935863. URL https://doi.org/10.1145/1935826.1935863.

Jianan Zhao, Meng Qu, Chaozhuo Li, Hao Yan, Qian Liu, Rui Li, Xing Xie, and Jian Tang. Learning on large-scale text-attributed graphs via variational inference, 2023. URL https://arxiv.org/abs/ 2210.14709.

Bin Zhou, Xiangyi Meng, and H. Eugene Stanley. Power-law distribution of degree–degree distance: A better representation of the scale-free property of complex networks. Proceedings of the National Academy of Sciences, 117(26):14812–14818, 2020. doi: 10.1073/pnas.1918901117. URL https: //www.pnas.org/doi/abs/10.1073/pnas.1918901117.

Jiong Zhu, Yujun Yan, Lingxiao Zhao, Mark Heimann, Leman Akoglu, and Danai Koutra. Beyond homophily in graph neural networks: Current limitations and effective designs. In NeurIPS, volume 33. Curran Associates, Inc., 2020.

Jiong Zhu, Ryan A. Rossi, Anup Rao, Tung Mai, Nedim Lipka, Nesreen . Ahmed, and Danai Koutra. Graph neural networks with heterophily. In Proceedings of the AAAI Conference on Artificial Intelligence, 2021.

A DETAILED RELATED WORK

Static GNN-LLM Fusion. Recent work has focused on two main paradigms for graph learning with LLMs: LLMs-as-Enhancers and LLMs-as-Predictors (Chen et al., 2024a; Li et al., 2024). For LLM-asEnhancers, LLMs generate enriched node features, ranging from simple text embeddings (Wu et al., 2024) to additional external knowledge (He et al., 2024; Chen et al., 2024a), effectively performing automated feature engineering to bolster GNN performance. For LLM-as-Predictors, the LLM replaces the GNN as the final classifier, processing serialized graph structure and text as prompts (Wu et al., 2024; Wang et al., 2025). Some variants retain a GNN encoder (Lin et al., 2025), but the LLM still produces the prediction. While both paradigms benefit from the reasoning of LLMs, they also suffer from distinct limitations. Enhancer methods are limited by their static, frozen features while also inheriting the inductive biases of the GNN, retaining the challenges seen with certain structural properties (e.g., heterophily). In contrast, LLM-as-Predictor methods can overcome the GNN inductive biases, but may also struggle to encode topology efficiently given prompt length limits and serialization mismatches (Firooz et al., 2025; Wang et al., 2025). Another common approach to fusing models together is through Mixture of Experts (MoE) which ensemble a set of models, each with expert knowledge. While prominent in LLM design (Cai et al., 2025) and graph MoE Wang et al. (2023), this strategy has yet to be applied to a mixture of LLMs and GNNs. Relatedly, LLMs have been shown capable to route to different GNNs (Jiang & Luo, 2025), yet, this still limits the pipeline to the inductive biases of the GNNs. Our approach adopts a hybrid paradigm that selectively invokes the LLM, using expert knowledge from both models, only when it is expected to help performance.

Adaptive GNN-LLM Fusion. A challenge for LLM-GNN models is high inference cost, especially when LLMs are used during training. While the static methods above are scalable by treating the LLM as a pre-processer, many methods consider using the LLM within their forward pass, calling the LLM across training and inference (Wu et al., 2024). While attempts to reduce LLM cost via neighborhood sampling or small language model distillation show promise (Fang et al., 2023; Chen et al., 2024a), the bottleneck remains where the LLM must be applied to all nodes. Only a few recent works explore adaptive GNN-LLM fusion, where the LLM is invoked selectively rather than universally. E-LLaGNN is one of the first in this space, selecting nodes for LLM augmentation using fixed heuristics based on graph properties such as degree, centrality, or text length (Jaiswal et al., 2024). While effective in some settings, these handcrafted rules require manual tuning and can vary in effectiveness across datasets. LOGIN (Qiao et al., 2025) introduces a more automated approach, using GNN uncertainty to identify challenging nodes to rewire and simplify message passing. However, this rewiring can disrupt structurally informative but challenging patterns, such as heterophilous links, which can be beneficial in real-world graphs (Luan et al., 2021). LLM-GNN is a label-free approach that leverages a similar principle as LOGIN, but uses clustering density as a proxy of difficulty as opposed to uncertainty (Chen et al., 2024b). Our approach differs from these methods in three key ways. First, while different metrics have been proposed to route, our systematic study identifies specific structural characteristics that benefit from LLM processing. Additionally, GLANCE does not require prespecified thresholds on these properties, instead learning how to utilize them to route. Second, instead of editing the underlying data to favor GNNs, we preserve its characteristics and invoke the LLM where the GNN struggles. Third, while LOGIN and LLM-GNN address non-differentiability by editing the underlying data to provide signal from the LLM, we retain the true graph signal and directly train the routing policy.

B LARGE LANGUAGE MODEL PROMPTING

In this section, we detail how we generate enhanced features for the base GNN models. Additionally, we provide prompts used to generate LLM embeddings within GLANCE. For both settings, we focus on the Qwen3 family of models, specifically using the Qwen3-Embed-8B model to generate embeddings.

B.1 LLM-AS-ENHANCER PROMPTS

We generate node-level text embeddings using Qwen3-Embed-8B by directly processing the node text (e.g., title and abstract for Cora). We use a max prompt length of 1024. Following the model’s official configuration, we use last-token pooling and then apply normalization. Concretely, for a batch of tokenized sequences with hidden states and (unpadded) last-token indices , the sentence

We keep Qwen’s include prompt behavior enabled, which prepends a short instruction prompt internally before the text. The resulting zi is used as an enhanced feature in place of the shallow features outlined above.

B.2 LOGIN PROMPTS

Following the format introduced in LOGIN, we construct prompts for each node by including the text of the ego node, a template that describes the graph and the instructions for the LLM, and the ground-truth labels and GNN predicted labels for nodes in the one-hop and two-hop neighborhood of the ego node.

LOGIN Example for Cora:

NODE 0: <ego-node text>

Given a citation graph NODE-IDX: <node_id>, NODE-LIST: [List of all node_id in two-hop neighborhood], ONE-HOP-NEIGHBORS: [List of node_id of one-hop neighbors], TWO-HOP-NEIGHBORS: [List of node_id of two-hop neighbors] NODE-LABELS: [List of ground truth labels] GNN-PREDICTED-NODE-LABEL: <GNN-predicted ego node label> where node 0 is the target paper and you see the true labels under ‘node_label‘.

Question: Which category does this paper belong to? Choose exactly one from [list of class labels].

Return JSON {{classification result: <choice>, explanation: <your reasoning>}} </END>

B.3 LLM-AS-PREDICTOR AND GLANCE PROMPTS

When using the LLM as predictor, we construct prompts for different hops of the neighborhood (ego-only, ego+1-hop, ego+2-hop) and encode them with Qwen3-Embed-8B. Doing this enables the LLM to retain per-hop information, a key design found in heterophilous graph learning, while also helping keep prompt lengths manageable. We utilize the same parameterizations as the LLM-as-Enhancer setting for each of these embeddings, but use a prompt length of 512 for the ego node, and 4096 for the oneand two-hop embeddings. Each prompt is formatted in a instruction–query layout compatible with Qwen3 embedding prompts and utilizes last-token pooling of the sequence.

The overall strategy in prompt design was to provide the classes and serialized neighborhood structure for a node. We do not over-engineer the prompt design, however, this can be one area for further opportunity in graph learning with LLMs. Additionally, providing additional context on a per-dataset basis (e.g. highlighting the domain of the dataset), could provide additional value.

Ego-Only Example:

Instruct: Predict the node’s category from the provided context. Possible categories: [list of class labels]. Query: EGO: <ego-node text> Category? </END>

Ego+1 Hop Example:

Instruct: Predict the node’s category from the provided context. Possible categories: [list of class labels]. Query: EGO: <ego-node text> HOP1: - <1 hop neighbor 1 text> - <1 hop neighbor 2 text>... Category? </END>

Ego+2 Hop Example:

Instruct: Predict the node’s category from the provided context. Possible categories: [list of class labels]. Query: EGO: <ego-node text> HOP2: - <2 hop neighbor 1 text> - <2 hop neighbor 2 text>... Category? </END>

As the degree of the datasets can be large, we use a neighbor sampling strategy during prompt construction to further maintain manageable prompt sizes. Specifically, we limit the size of each neighborhood to be up to 5 neighbors per node. Thus, the ego-only text contains one node, the ego + 1-hop can contain up to 6 nodes (1 + 5 neighbors), and the ego + 2-hop can contain up to 26 nodes (1 + 5*5). We use a prompt length of 1024 for the ego-node, and 4096 for the ego + 1-hop and ego + 2-hop. On a per-node basis, text is capped to a fixed character budget (2000 characters). We process the embeddings using normalization before concatenation.

When originally fine-tuning the LLM (such as the results in Figure 1, we apply a 2-layer MLP to map the concatenated embeddings to the label space for each model. Then, when incorporating the pre-trained LLM into GLANCE, we remove the MLP head and use the concatenated embeddings directly with the refiner MLP.

C MODEL TRAINING AND HYPERPARAMETERS

We describe here the training protocols for the base GNNs, the LLM-as-Predictor baseline, and our proposed GLANCE model. Across all models, we tune the learning rate from and the weight decay from . We apply gradient clipping at for all gradients g. Training is performed on a single NVIDIA A40 GPU using the AdamW optimizer, with dataset splits of 50/25/25 for train/validation/test. For OGB-Products, we cap the train set to 50,000 nodes for GLANCE, but retain the same test set with the GNN models.

C.1 GNN (AND MLP) TRAINING

For the GNN backbones, we use implementations integrated into PyTorch Geometric when available. Otherwise, for GBK-GNN 1 and GGCN2 we use their official implementations. For all GNNs, we use a hidden dimension of 64 and a batch size of 128. We search over network depths of layers. Training is run for up to 1000 epochs, with early stopping applied if the validation accuracy does not improve for 30 consecutive epochs. Each model includes a one-layer MLP projection head, which both

1https://github.com/Xzh0u/GBK-GNN 2https://github.com/Yujun-Yan/Heterophily_and_oversmoothing

produces the final predictions and allows the learned GNN embeddings to be reused within GLANCE. For heterophilous architectures such as FAGCN, GGCN, and GBK-GNN, we adopt the hyperparameter settings recommended in their official repositories. For the MLP local homophily estimator, we follow a similar strategy as the GNNs, using a depth of 2 and hidden dimension of 64. Both the GNN and MLP additionally use ReLU activations and a dropout of 0.2. For OGB-Products, we limit to 20 epochs with a patience of 5, and limit the neighbor sampling rate to 30 nodes per neighbor. Additionally, we highlight that the official implementation of GGCN runs out of memory for OGB-Products due to requiring access to the full graph upon initialization. Even with sparse tensors, this is found to be impossible for the OGB-Products dataset.

C.2 LOGIN TRAINING

For LOGIN, we use GNN backbones with setups described in the previous section. We then estimate the uncertainty of nodes by doing five forward passes with random dropout at a 0.3 dropout rate. We take the top nodes with defined by:

to query the LLM, where is the number of nodes and is the heterophily ratio. While LOGIN originally utilizes the heterophily ratio to query nodes, we find this to be too large for Arxiv23 and causes large training times. Thus, the 1000 node cap allows us to utilize LOGIN for Arxiv23. We use a pretrained Vicuna-7B to generate the predictions and explanations. Depending on whether the LLM predict labels match the ground-truth labels, we either append the explanation to the original node text or prune edges in the node’s one-hop neighborhood as implemented in LOGIN. We use a similarity threshold of 0.15 for pruning the edges of nodes whose LLM prediction does not match the ground-truth label. Finally, a pretrained E5-Large model is used to encode the new set of node text that includes the original text and LLM explanations.

C.3 LLM TRAINING

For the LLM baseline, we fine-tune Qwen3-Embed-8B to serve as an LLM-as-Predictor. To reduce both memory and compute, we employ parameter-efficient adaptation via Low-Rank Adaptation (LoRA), inserting adapters into the attention projection layers with a rank of 16 and scaling factor . We also enable FlashAttention-2 and use bfloat16 precision. During training, only LoRA parameters are updated while the base model remains frozen.

We use a batch size of 1 with gradient accumulation over 8 steps, yielding an effective batch size of 8. Training proceeds for a maximum of 10 epochs with early stopping applied if validation accuracy fails to improve for 2 epochs. Since training on all available nodes is infeasible, we cap the training set at 3,000 randomly sampled nodes. As with the GNNs, we include a one-layer MLP head to project embeddings into the label space.

C.4 GLANCE TRAINING

When training GLANCE, we first independently train a GNN, LLM, and shallow MLP via the training processes explained in Section C.1 and Section C.3. Specifically, each backbone model is trained to optimize the node classification task and is then frozen to ensure its modeling capabilities are maintained within GLANCE. Thus, when training GLANCE, only the router and the refiner models that fuse the two pathways are updated. The router is implemented as a logistic regression model to predict the likelihood of routing, while the refiner is a 2-layer MLP following the same configuration as the MLP local homophily estimator. Routing decisions are made by selecting the top- nodes per batch according to the router’s predicted scores. To gradually reduce reliance on the LLM during training, we decay the routing budget across epochs using an exponential schedule:

where equals the batch size, is the ending routing budget for the last epoch, and r is the decay factor. We set =/4 and during our experiments, requiring significantly less LLM calls across training.

We train with mixed-precision using bfloat16, and since no gradients are propagated through the LLM, the memory footprint is greatly reduced, allowing us to use a batch size of 32. For the different datasets, we

set equal to the batch size. As in the LLM baseline, we cap the training set at 3,000 nodes. Early stopping is applied with a patience of 2 epochs. We additionally tune the LLM query cost ∈{0.1,0.2,0.3}, and set the router weight =1.0 and entropy regularization weight =0.01 by default.

D DATASET DETAILS

To characterize the limitations of current heuristics to route, as well as evaluate GLANCE, we utilize three widely studied TAG benchmarks: Cora (Mccallum et al., 2000), Pubmed (Sen et al., 2008a), and Arxiv23 (He et al., 2024). Then, for our study on scalability, we utilize OGB-Products (Hu et al., 2020), representing a multiple order of magnitude increase in dataset size compared to typically studied TAGs. These datasets also differ significantly in homophily levels, allowing us to probe routing decisions under diverse conditions.

• Cora. Cora is a citation network consisting of machine learning papers. Papers share an edge if they share a citation. The data is from https://github.com/myflashbarry/ LLM-benchmarking (Wang et al., 2025). The canonical feature vector for each paper is a bag-of-words feature vector. The nodes are categorized into 7 different AI subfields. Cora possesses the highest global homophily level out of the datasets ().

• Pubmed. Pubmed is a larger citation graph on biomedical articles, where papers share an edge if they share a citation. The data is from https://github.com/myflashbarry/ LLM-benchmarking. Each article is represented by a TF–IDF word vector, with labels corresponding to 3 different disease types. Similar to Cora, Pubmed exhibits high global homophily (), but also posseses a larger pocket of heterophilous nodes (see Figure 1).

• Arxiv23. Arxiv23 is a recently curated citation subgraph of arXiv covering computer science papers published after 2023, surpassing the knowledge cutoff of flagship models like ChatGPT- 3.5. The data is from https://github.com/XiaoxinHe/tape_arxiv_2023. We find that the original proposed Arxiv23 dataset has a large number of isolated nodes, which can confound the evaluation metrics given over half of the nodes do not possess a graph structure. Thus, we subset Arxiv23 to only include the largest connected component. Each article is represented by a Word2vec embedding, with each class corresponding to a computer science subfield. Unlike Cora and Pubmed, Arxiv23 displays a wider array of homophilous and heterophilous edges, with (), making it a challenging benchmark for neighborhood-averaging GNNs.

• Arxiv-Year Arxiv-Year is a heterophilous variant of OGB-Arxiv, using the publication year as the label. The data is from https://ogb.stanford.edu, but with the labels created via quantile ranges (see https://github.com/CUAI/Non-Homophily-Large-Scale/ blob/master/data_utils.py, producing 5 labels. The dataset is roughly an order of magnitude larger compared to the three previous datasets. For this dataset, we only use enhanced features from the raw text. Arxiv-year is heterophilous ().

• OGB-Products OGB-Products is a large-scale e-commerce TAG provided from the OGB https://ogb.stanford.edu. The task is to predict the category of e-commerce items, where items share a link if co-purchased. The dataset is roughly two orders of magnitude larger than the TAGs seen in previous literature, with 47 classes. We only use enhanced features for this dataset. OGB-Products is homophilous ()

The explicit number of features, classes, and other key metrics are summarized in the table below. Notably, the datasets we study contain a wide variety of sizes, densities, and homophily levels.

E ROUTING FOR GCN AND GCNII

In addition to our enhanced results provided in the main section of the paper, we include supplemental results with routing on the base GCN and GCNII architectures with their original features.

E.1 ROUTING WITH PRIOR METHODS

Similar to the results seen on the enhanced model variants, we see that in Table 7, across Cora, Pubmed, and Arxiv23, uncertainty-based routing consistently delivers the strongest NCS among the heuristic baselines for both backbones, with degree and C-density lagging behind (often near or below the random baseline on Cora). This result confirms our original finding that, when routing into an LLM, predictive uncertainty is the most reliable heuristic from the prior methods, while the other signals alone can even harm performance.

E.2 HOMOPHILY-BASED ROUTING (ESTIMATED VS. TRUE)

We begin by evaluating the effectiveness of the true local homophily, . We can see in Table 8, similar to the main text, achieves the highest NCS across models and datasets. However, when we move to estimated homophily , we can see that it follows a similar relative trend. Relative degree ¯ maintains its relatively weaker performance. These findings further confirm that homophily is a powerful routing signal, with conferring robust gains.

Figure 4: Stratified Performance Based on Homophily and Degree. Darker red denotes instances where the LLM performs best, and darker blue denotes instances where the GNN performs best. Across methods, we see further deviation in performance as compared to the individual metrics

E.3 CORRELATION BETWEEN HOMOPHILY AND UNCERTAINTY

Table 9 shows a moderate correlation (inverse) between model uncertainty and local homophily across all datasets. Intuitively, as homophily decreases (i.e., more label conflict in neighborhoods), uncertainty increases. This empirically links the two most effective routing cues where regions of low homophily tend to be high-uncertainty. However, this correlation is relatively weak, thus we hypothesize including both metrics can provide complimentary information, i.e. the signals are not providing redundant information.

E.4 STRATIFIED ANALYSIS ON PUBMED

In Figure 5, we observe consistent trends with those reported on Cora and Arxiv23. Specifically, while performance across models converges in the high homophily regime, LLM models deliver pronounced gains in the low-homophily and low-degree settings, outperforming strong GNN backbones by substantial margins. For example, LLM-based predictions exceed the next best GNN-enhanced variant by as much as 10.4% on heterophilous nodes ( <0.20). These findings further confirm that the advantage of LLM augmentation is not dataset-specific, but generalizes across datasets, highlighting homophily as a reliable signal for routing.

E.5 HOMOPHILY AND DEGREE INTERPLAY

While homophily and degree display diverging trends between LLM and GNN training, highlighting their complimentary strengths, it also known that their interplay can further inform GNN behavior. In Figure 4, we stratify across both properties for each model and dataset, further demonstrating increased divergence in

Figure 6: Stratified Performance for Different at Test-Time. We use a batch size of 32 when testing GLANCE, and use different routing budgets depending on . We find that performance tends to increase for heterophilous groups of nodes as we increase , demonstrating GLANCE’s ability to take advantage of larger routing budgets.

performance differences. For instance, while Pubmed attains a 10% difference when looking at homophily in isolation, this difference can go as high as ∼30% under the same setting (e.g., for GCNII (enhanced) on Cora, low homophily and low degree can experience a significant improvement as compared to high homophily and high degree).

F ADDITIONAL ABLATION AND SENSITIVITY ANALYSES

In this section, we present supplemental analyses introduced in Section 6.3. We begin by examining the sensitivity of GLANCE to the parameter , which controls the number of nodes routed to the LLM at test time. This analysis reveals how performance changes as the budget for LLM queries is adjusted, offering insight into the trade-off between predictive accuracy and computational cost. Understanding this trade-off is particularly important in practice, as practitioners may wish to tune to align with real-world latency or cost constraints. Then, we ablate the routing features used by GLANCE to determine which signals are most critical for effective routing. By systematically removing each feature, we isolate their individual contributions to routing accuracy and downstream performance. Together, these supplemental analyses deepen our understanding of GLANCE’s behavior, confirming both its robustness to different query budgets and its reliance on principled routing features that target GNN failure regimes.

Figure 5: Stratified Performance. Performance is given for local homophily (top) and relative degree (bottom); bars denote property distributions (right y-axis).

F.1 PERCENT CHANGE DURING ROUTING

As introduced in Section 6.3, we analyze the effect of varying the routing budget on GLANCE’s performance. Figure 6 reports stratified results across different levels of local homophily, allowing us to assess how additional LLM queries impact different subpopulations of nodes. We observe that increasing generally improves performance on heterophilous and low-homophily nodes, where the GNN struggles most. Importantly, with , GLANCE consistently achieves the highest overall performance across routing strategies, demonstrating that allocating a larger query budget enables the framework to more effectively target the nodes where LLM assistance is most beneficial. This highlights a practical trade-off where larger increases computational cost but can yield substantial accuracy improvements, particularly in challenging structural regimes.

Full Routing. To further illustrate the value of selective routing, we evaluate the setting where every node in a batch is routed to the LLM (denoted as =32 for a batch size of 32), allowing the LLM to influence all predictions. As shown in Table 10, full routing is not necessarily beneficial, reinforcing the need for GLANCE’s selective strategy. Specifically, while a higher routing budget improves performance on the most challenging nodes (those in the lowest homophily bins) we also observe a decline in accuracy on high homophily nodes. Consequently, both Cora and Arxiv23 see reduced overall accuracy despite the substantially higher query cost. This trend highlights an important trade-off: although LLMs can correct difficult nodes, indiscriminately routing all nodes dilutes accuracy where the GNN already performs well and significantly increases computational cost. In contrast, GLANCE’s selective routing (e.g., =12) delivers substantial gains on heterophilous nodes while preserving strong performance on high homophily regions, all with far fewer LLM calls. We also note that while Pubmed does not exhibit as pronounced of a drop, this result is expected as Pubmed’s node text has been shown to be highly informative, even enabling LLMs to predict node labels accurately in zero-shot settings (Chen et al., 2024a). However, despite this capability, the performance gap between =12 and =32 remains extremely small in the highest-homophily bin, indicating that the significant increase in computation offers minimal benefit.

F.2 GRANULAR ANALYSIS ON ROUTING PERFORMANCE

In this section, we provide a deeper analysis on the performance differences between routed and non-routed nodes. Moreover, we offer additional studies to characterize performance under a random router.

F.2.1 PERFORMANCE IMPROVEMENTS FROM LLM ROUTING

To characterize the effectiveness of GLANCE’s routing, we evaluate the predictive accuracy specifically on the subset of nodes routed to the LLM, denoted R. We compare the performance of GLANCE against a baseline where predictions for this exact same set R are made solely by the GCNII backbone. This direct comparison provides clear evidence for whether invoking the LLM improves prediction accuracy for the nodes selected by our router. The results are shown in Table 11.

Findings. We compare the accuracies on the routed nodes R using predictions from the GCNII backbone versus the LLM. Across all datasets, routing these selected nodes to the LLM yields substantially higher

accuracy. Specifically, GLANCE improves performance on by +4.9% (Cora), +6.4% (PubMed), and +4.5% (Arxiv23). These significant gains demonstrate that GLANCE’s routing policy is highly effective at identifying challenging nodes that directly benefit from the enhanced capabilities of the LLM.

F.2.2 RANDOM ROUTING PERFORMANCE

To contextualize the routing improvements further, we evaluate a random routing baseline that, for the same query budget ( =12), selects nodes at random while keeping all other components of GLANCE unchanged. This experiment serves as a natural lower bound on routing quality and serves as another indicator to whether GLANCE is able to identify difficult nodes. We measure the difference in overall, routed, and non-routed performance between the standard and random routing schemes in Table 12 where more positive is better. We perform 3 runs per experiment and report standard deviations.

Findings. We first compare GLANCE’s learned router (denoted Standard) against a random routing baseline. Under random routing, overall performance degrades across all datasets, with accuracy dropping by up to 3.1% (Cora). More importantly, accuracy on the routed subset R is generally lower than with the learned router, suffering drops of 4.3% and 2.3% on Cora and Arxiv23, respectively. This result demonstrates that randomly selected nodes benefit less from LLM refinement. While PubMed exhibits a marginal 0.3% increase on R under random routing, the non-routed accuracy drops consistently across all datasets. This indicates that a random policy frequently wastes LLM queries on simpler nodes that the base GNN could already classify correctly, leaving behind challenging nodes that the GNN struggles to resolve. Together, these results reinforce that GLANCE’s lightweight routing reliably identifies semantically and structurally difficult nodes, yielding targeted performance gains where standard GNNs fail.

Figure 7: Ablation Study Over Routing Features - Overall Performance. Each plot denotes the performance changes, relative to full GLANCE performance, when training without one of the routing features. Performance typically decays across datasets and features, highlighting the benefit of each feature for GLANCE’s robust performance.

Figure 8: Ablation Study Over Routing Features - Stratified Performance. Each set of bars denotes the stratified performance of GLANCE when training without one of the routing features. The gray bar denotes full GLANCE training. We find that the largest performance drops occur in the heterophilous regions, highlighting that GLANCE is specifically targeting these difficult nodes.

F.3 ROUTING FEATURE SENSITIVITY

While each routing feature is motivated either by prior work or our analysis in Section 4.1, we quantify their contribution through an ablation study. For each dataset, we retrain GLANCE while removing each feature in turn and measure the resulting change in accuracy. Figure 7 reports the performance drop relative to the full model, showing that overall accuracy consistently declines when any feature is removed. Notably, the impact varies across datasets where no single feature dominates universally, underscoring that different structural signals matter in different settings. This trend is further evident in Figure 8, where, within stratified bins, removing a single feature produces substantial degradations in performance.

F.4 DIFFERENT ROUTING PENALTIES

In this section, we provide an additional sensitivity analysis on , GLANCE’s penalty term. Increasing , the margin a routed node must surpass for the LLM to be selected, systematically re-allocates LLM usage toward the hardest, most heterophilous nodes. As shown in Table 13, raising improves accuracy in the lowest-homophily bin across Cora (from 45.1 to 51.0), Pubmed (from 68.0 to 71.3), and Arxiv23 (from 42.4 to 45.5), while leaving the most homophilous bin effectively unchanged and producing only small drops in the intermediate bins. In practice, the router spends fewer calls on marginal gains in the intermediary homophily levels where signal is weaker and concentrates budget where the GNN can be most improved upon. This behavior makes a simple, yet effective, knob to trade-off homophilous and heterophilous nodes. When leveraging GLANCE, we recommend tuning on validation metrics in the range (0.1-0.3) to determine the most effective value.

G SCALING GLANCE TO LARGER DATASETS

Having evaluated GLANCE’s performance on smallto medium-scale TAG benchmarks, we now turn to scalability. A concern with LLM–GNN fusion is whether additional components introduced by a method introduce significant overheads, especially when moving to much larger datasets. To address this, we first break down the runtime of GLANCE into its major components: (i) GNN computation, (ii) LLM computation, and (iii) GLANCE-specific modules, namely the router, feature generation, and refinement stages. As shown in Figure 9, the results reveal a clear trend where the LLM dominates the overall runtime,

the GLANCE-specific components are extremely lightweight, constituting only a negligible fraction of total cost. Both routing and refinement involve simple feed-forward operations over low-dimensional features, while feature generation is largely pre-computed within the pipeline. This finding has two key implications. First, it validates our design choice to offload expensive reasoning exclusively to the LLM, while keeping routing and refinement efficient. Second, it demonstrates that scaling GLANCE to larger graphs is primarily limited by the LLM’s cost, rather than by the fusion mechanism itself. We therefore use this as motivation to apply GLANCE to the substantially larger dataset OGB-Products seen in Table 5, showcasing its ability to scale while retaining efficiency and accuracy.

Figure 9: Runtime breakdown of GLANCE. LLM computation dominates overall runtime, while GLANCE-specific modules add only negligible overhead, confirming the framework’s scalability to larger datasets.

Computational Cost. We analyze the computational complexity of GLANCE by isolating the costs of its three key components: (1) the routing features, (2) the router, and (3) the refiner. As GLANCE operates on top of frozen GNN and LLM encoders and is agnostic to their internal architectures, we do not define a specific GNN or LLM cost. Instead, we focus on the GLANCE-specific elements. For each component, we denote the cost of a forward and backward pass, when applicable, by and , respectively.

Notation. Let denote the batch size, the routing budget per batch, the dimensionality of the routing feature vector, the number of classes for a dataset, and the maximum degree of any node in the batch. We also define the following per-node costs:

• : cost of applying the frozen GNN encoder to one node,

• : cost of an LLM forward pass for one routed node,

• : cost of applying the homophily MLP to one node,

• : cost of applying the refiner MLP to one routed node.

Building Routing Features (Per Batch). Routing features consist of: (1) GNN embeddings, (2) GNN uncertainty, (3) degree information, (4) original node features, and (5) homophily estimates. Using the above per-node costs, this results in the per-batch cost:

Router. The router applies a linear transformation to the features and selects the top-k nodes to route:

Selective LLM Queries. GLANCE queries the LLM for the k routed nodes (with no backward pass):

Refiner MLP. For each routed node, GLANCE applies the refiner MLP:

Interpretation. GLANCE’s complexity is comprised of two core terms: (1) the full batch processing that creates routing features leveraging the GNN, and (2) the LLM and refiner computation applied only to the routed nodes. Given far exceeds the other terms (as shown in Figure 9), one can scale up the GNN, router, or refiner without incurring large relative increases in runtime. Moreover, because training uses a policy-gradient-style optimization procedure, the cost differential between training and inference is small as the backward pass is only on the cheaper components of the pipeline.