Multi-Feature Fusion Strategies for Enhancing Knowledge Graph Embedding
多特征融合策略在知识图谱嵌入中的应用

Knowledge graph  知识图谱
Entity-relation interactions
实体-关系交互
Feature fusion  特征融合
Convolutional neural network
卷积神经网络
Link prediction  链接预测

Knowledge graphs (KGs) have become a widely adopted data model to structurally represent knowledge and its relationships, with applications in areas such as semantic search [34], recommendation systems [32], natural language processing (NLP) [8], and intelligent question answering [36].By establishing a multi-layered semantic network wherein nodes represent entities and edges denote relationships, KGs are capable of effectively capturing and representing the intricate semantic and structural interconnections among entities.
知识图谱（KGs）已成为一种广泛采用的数据模型，用于结构化表示知识及其关系，其应用领域包括语义搜索[34]、推荐系统[32]、自然语言处理（NLP）[8]和智能问答[36]。通过构建多层语义网络，其中节点代表实体，边表示关系，知识图谱能够有效捕捉和表示实体之间复杂的语义和结构关联。

In real-world applications, most knowledge graphs are often incomplete. Chain-based prediction, as an important reasoning method, infers missing information by utilizing the existing knowledge within the graph, thereby filling in the gaps. KGE techniques map entities and relations to a lowdimensional vector space, enabling semantic information within the graph to be computed and reasoned in the form of vectors. This provides effective support for chain-based prediction, further promoting the inference of missing parts of the graph. Early KGE methods primarily focused on geometric transformation mechanisms, such as TransE [5], TransH [33], and TransD [10], which capture the relational structures between entities through simple spatial transformations. RotatE [24] replaces translation with rotation, enabling the model to capture a wider range of relationship patterns. Meanwhile, traditional bilinear scoring functionbased methods, such as DistMult [35] and ComplEx [28],
在实际应用中，大多数知识图谱往往是不完整的。链式预测作为一种重要的推理方法，通过利用图中已有的知识来推断缺失的信息，从而填补空白。知识图谱工程（KGE）技术将实体和关系映射到低维向量空间，使图中的语义信息能够以向量的形式进行计算和推理。这为链式预测提供了有效的支持，进一步促进了图中缺失部分的推断。早期 KGE 方法主要关注几何变换机制，如 TransE[5]、TransH[33]和 TransD[10]，通过简单的空间变换捕获实体间的关联结构。RotatE[24]将平移替换为旋转，使模型能够捕获更广泛的关系模式。与此同时，传统的双线性评分函数方法，如 DistMult [35]和 ComplEx [28]，
effectively model the interactions between entities and relations. The RESCAL [20] uses a matrix to represent relations and captures the latent interactions between entities. However, these methods exhibit certain limitations when handling asymmetric relations. With advancements in deep learning technologies, the Convolutional Neural Network (CNN) [17] has been increasingly applied to knowledge graph reasoning tasks. Recent research has explored the use of neural network architectures for KGE, where ConvE [9] employs convolutional filters to capture local interaction patterns, and R-GCN [21] utilizes a relational graph convolutional network to aggregate relational information.
有效建模实体与关系之间的交互。RESCAL [20] 采用矩阵表示关系，并捕捉实体之间的潜在交互。然而，这些方法在处理非对称关系时存在一定局限性。随着深度学习技术的进步，卷积神经网络（CNN）[17] 逐渐被应用于知识图谱推理任务。近期研究探索了神经网络架构在知识图谱表示中的应用，其中 ConvE [9] 通过卷积滤波器捕捉局部交互模式，而 R-GCN [21] 则利用关系图卷积网络聚合关系信息。

Although neural network-based knowledge graph embedding (KGE) methods have made significant strides in recent years, existing embedding models still face several critical challenges. First, current KGE models predominantly focus on direct entity-relation pairs, failing to fully exploit the rich structural information embedded within local neighborhoods. This limitation restricts the model’s ability to capture complex topological patterns, which are essential for understanding the semantics of entities within the context of relations.Traditional approaches rely on static, low-dimensional representations, which are inadequate for encoding the multi-dimensional semantic nature of knowledge graphs. In knowledge graphs, the semantic features of entities and relations are dynamic, influenced by their structural context and co-occurrence patterns. However, static representations are inherently limited in capturing the dynamic evolution of semantic information. Furthermore, existing methods lack sophisticated mechanisms for integrating heterogeneous semantic features. Current models are
尽管基于神经网络的知识图谱嵌入（KGE）方法在近年来取得了显著进展，但现有的嵌入模型仍面临多个关键挑战。首先，当前的 KGE 模型主要关注直接的实体-关系对，未能充分利用局部邻域中蕴含的丰富结构信息。这一局限性限制了模型捕捉复杂拓扑模式的能力，而这些模式对于理解实体在关系上下文中的语义至关重要。传统方法依赖于静态、低维的表示，无法有效编码知识图谱的多维语义特性。在知识图谱中，实体和关系的语义特征是动态的，受其结构上下文和共现模式的影响。然而，静态表示在捕获语义信息动态演变方面存在固有局限性。此外，现有方法缺乏整合异构语义特征的复杂机制。当前模型是
often unable to effectively merge these different types of features, resulting in a narrow and incomplete semantic representation. Knowledge graphs typically embody complex and diverse semantics, and developing an embedding model that can effectively integrate these heterogeneous characteristics remains a significant challenge, especially as the scale and heterogeneity of knowledge graphs continue to grow.
通常难以有效地融合这些不同类型的特征，导致语义表示过于狭窄且不完整。知识图谱通常包含复杂多样的语义，而开发一种能够有效整合这些异构特征的嵌入模型仍是一项重大挑战，尤其随着知识图谱的规模和异构性持续增长。

To address these limitations, we propose a novel MultiFeature Fusion Embedding (MFFE) framework that systematically integrates multiple complementary feature representations through specialized neural modules. Our approach recognizes that effective knowledge graph embedding requires the coordinated modeling of diverse structural and semantic aspects, rather than relying on single-feature extraction strategies. We design four core modules: RCNA, DRM, ERAM, and AFFM. Specifically, RCNA constructs a local subgraph tensor and applies a relation-aware convolutional transformation to aggregate neighborhood features. DRM introduces a learnable relation matrix that dynamically modulates the influence of each relation on entities, thereby enhancing the model’s expressive capability. ERAM utilizes multi-head attention to capture higher-order interaction features between entities and relations. AFFM employs dynamic bidirectional gated fusion mechanism to fuse neighborhood features, relation-aware features, and entityrelation interaction features, generating high-quality embedding representations.
为了解决这些局限性，我们提出了一种新型的多特征融合嵌入（MFFE）框架，该框架通过专门的神经网络模块系统性地整合多种互补的特征表示。我们的方法认识到，有效的知识图谱嵌入需要对多样化的结构和语义方面进行协同建模，而非依赖单一特征提取策略。我们设计了四个核心模块：RCNA、DRM、ERAM 和 AFFM。具体而言，RCNA 构建局部子图张量，并应用关系感知卷积变换聚合邻域特征。DRM 引入可学习的关系矩阵，动态调节每个关系对实体的影响，从而提升模型的表达能力。ERAM 通过多头注意力机制捕获实体与关系之间的更高阶交互特征。AFFM 采用动态双向门控融合机制，融合邻域特征、关系感知特征及实体-关系交互特征，生成高质量的嵌入表示。

The contributions of this research are:
本研究的贡献包括：

Neural network-based models leverage deep learning techniques to capture complex interactions between entities and relations in knowledge graphs. One of the pioneering models in this category is ConvE, which uses CNN to apply 2D convolutions to the interaction between entities and relations, extracting richer features from these interactions. InteractE [29] uses a checkerboard reshaping method to enable complete interactions between entities, further enhancing the feature extraction process. HypER [3] use hypernetwork models that share weights across layers and adaptively combine them based on inputs, allowing for more flexible entityrelation interactions. In ConvR [12], relations are utilized
基于神经网络的模型利用深度学习技术捕捉知识图谱中实体与关系之间的复杂交互。该领域开创性模型之一是 ConvE，其通过卷积神经网络（CNN）对实体与关系之间的交互应用二维卷积，从而从这些交互中提取更丰富的特征。InteractE [29]采用棋盘重塑方法实现实体之间的完全交互，进一步提升特征提取效果。HypER [3] 采用超网络模型，该模型在层间共享权重并根据输入自适应地组合权重，从而实现更灵活的实体-关系交互。在 ConvR [12] 中，关系被用于
directly as convolutional kernels, enabling explicit interactions between entities and relations, and thereby enhancing the model’s ability to capture relational patterns. M-DCN [40] extracts interaction features of entities and relations through multi-scale convolutional kernels, enhancing the expressiveness of the embeddings. JointE [41] combines 1D convolution and 2D convolution to promote entity-relation interaction, and dynamically generates convolutional kernels based on internal embeddings. In MSHE [11] entities, relationships, and joint knowledge sources are constructed via a base multi-source extension framework, followed by the use of hierarchical convolutional networks to capture multilevel semantic features. CALP [39] fuses 2D convolution and self-attention mechanisms to simultaneously capture local and global feature interactions between entities and relations, significantly improving link prediction performance while maintaining low-dimensional embeddings. DTAE [7] constructs adaptive convolutional kernels based on relation representations and combines multidimensional attention mechanisms to capture nuanced interactions between entities and relations, while employing dynamic adaptive atrous convolutional networks to expand the receptive field for obtaining global contextual information. Recent studies, such as MGIF [15], have made significant contributions to multi-perspective knowledge graph embedding by integrating global and interaction features. MGIF primarily extracts features through global entity-sharing and relationsharing convolution operations, which struggle to fully capture the relationship-specific neighborhood patterns crucial for complex semantic modeling. Additionally, its feature fusion approach mainly relies on self-attention mechanisms to combine different perspective features. In contrast, the MFFE framework introduces a more comprehensive feature extraction strategy, leveraging neighborhood aggregation, dynamic relation-awareness, and interaction information to capture a more diversified and granular set of semantic features. The adaptive bidirectional gated fusion mechanism employed by MFFE allows for more precise integration of heterogeneous features compared to attention-based fusion methods.
直接用作卷积核，使实体与关系之间能够明确交互，从而提升模型捕捉关系模式的能力。M-DCN [40] 通过多尺度卷积核提取实体与关系的交互特征，提升嵌入的表达能力。JointE [41] 结合 1D 卷积与 2D 卷积促进实体-关系交互，并基于内部嵌入动态生成卷积核。在 MSHE [11]中，通过基础多源扩展框架构建实体、关系及联合知识源，随后采用分层卷积网络捕获多层次语义特征。CALP [39]融合 2D 卷积与自我注意机制，同时捕获实体与关系间的局部与全局特征交互，显著提升链接预测性能并保持低维嵌入。DTAE [7] 基于关系表示构建自适应卷积核，结合多维注意力机制捕捉实体与关系间的细微交互，并采用动态自适应空洞卷积网络扩展受容野以获取全局上下文信息。近期研究，如 MGIF [15]，通过整合全局与交互特征，为多视角知识图谱嵌入做出了重要贡献。MGIF 主要通过全局实体共享与关系共享卷积操作提取特征，但难以充分捕捉复杂语义建模所需的关系特异性邻域模式。 此外，其特征融合方法主要依赖于自注意力机制来整合不同视角的特征。相比之下，MFFE 框架引入了一种更全面的特征提取策略，通过邻域聚合、动态关系感知和交互信息，捕获更丰富多样且细粒度的语义特征集。MFFE 采用的自适应双向门控融合机制，相较于基于注意力的融合方法，能够更精准地整合异构特征。

Graph Convolutional Networks (GCNs) leverage structural information from knowledge graphs to learn more expressive embeddings, demonstrating particular effectiveness in modeling complex relational data. The R-GCN enhances multi-relational data processing by assigning distinct weights to different relation types, thereby capturing diverse relational patterns. CompGCN [30] proposes various composition operations for neighbor aggregation to model the structural patterns of multi-relational graphs. KE-GCN [38] proposes a joint learning framework that integrates knowledge embedding with graph convolution, simultaneously updating both entity and relation embeddings. This approach overcomes the limitations of traditional GCNs, which are typically restricted to homogeneous graphs or focus solely on node embeddings. It offers a unified solution for joint modeling of entities and relations in heterogeneous knowledge graphs. SHGNet [18] removes the transformation
图卷积网络（GCNs）利用知识图谱中的结构信息学习更具表达力的嵌入，在建模复杂关系数据方面表现出特别的有效性。R-GCN 通过为不同关系类型分配不同的权重，提升了多关系数据处理能力，从而捕捉多样化的关系模式。CompGCN [30]提出了多种邻居聚合的组合操作，以建模多关系图的结构模式。KE-GCN [38] 提出了一种联合学习框架，将知识嵌入与图卷积相结合，同时更新实体和关系嵌入。该方法克服了传统 GCN 的局限性，后者通常仅适用于同质图或仅关注节点嵌入。它为异构知识图谱中实体与关系的联合建模提供了统一解决方案。SHGNet [18] 消除了转换

Fig.1. Overall pipeline of the MFFE framework for knowledge graph embedding.
图 1. MFFE 框架用于知识图谱嵌入的整体流程图。
matrix and nonlinear activation functions from traditional GNNs, retaining only the essential neighborhood aggregation operation and incorporating relation features into the feature propagation process, while selectively aggregating informative features through node aggregation and relation weighting mechanisms. MGTCA [22] employs a mixed geometry message function to integrate information from hyperbolic space, hypersphere space, and Euclidean space to generate rich neighbor messages, and designs a trainable convolutional attention network to achieve autonomous switching of graph neural network types.
矩阵和非线性激活函数，仅保留传统 GNN 中的核心邻域聚合操作，并将关系特征融入特征传播过程，同时通过节点聚合和关系权重机制有选择性地聚合信息丰富的特征。MGTCA [22] 采用混合几何消息函数整合双曲空间、超球空间和欧几里得空间的信息以生成丰富的邻域消息，并设计可训练的卷积注意力网络实现图神经网络类型的自主切换。

Transformer architectures have recently been adapted for KGE. SDFormer [16] incorporates Transformer architectures to further enhance entity-relation interactions through attention mechanisms. KG-BERT [37] reformulates knowledge graph triplets as textual sequences and leverages pretrained language models for knowledge graph completion. StAR [31] introduces a structure-augmented text representation learning framework that optimizes both structural and semantic features for efficient knowledge graph reasoning. Relphormer [4] introduces the Triple2Seq mechanism to dynamically sample context subgraph sequences and incorporates a structurally enhanced self-attention mechanism to retain the structural information of KGs. This effectively addresses the heterogeneity issue faced by traditional Transformers in handling KGs. LP-BERT [14] is a multi-task pre-training framework for knowledge graph completion that combines three pre-training tasks: Masked Language Model, Masked Entity Model, and Masked Relation Model, to simultaneously learn contextual information and structural knowledge of triples.
变压器架构最近被适应于知识图谱表示（KGE）。SDFormer [16] 通过注意力机制将变压器架构融入其中，进一步增强实体关系交互。KG-BERT [37] 将知识图谱三元组重新表述为文本序列，并利用预训练语言模型进行知识图谱补全。StAR [31] 提出了一种结构增强的文本表示学习框架，通过优化结构和语义特征，实现高效的知识图谱推理。Relphormer [4] 引入了 Triple2Seq 机制，动态采样上下文子图序列，并整合了结构增强的自我注意力机制以保留知识图谱的结构信息。这有效解决了传统 Transformer 在处理知识图谱时面临的异构性问题。LP-BERT [14] 是一个知识图谱补全的多任务预训练框架，结合了三个预训练任务：遮蔽语言模型、遮蔽实体模型和遮蔽关系模型，以同时学习三元组的上下文信息和结构知识。

In this section, we first delineate the task formulation for addressing the link prediction problem in KGs, and then propose a methodology based on multi-feature fusion embeddings. Fig. 1 provides an overview of our proposed MultiFeature Fusion Embedding (MFFE) framework, which extracts diverse features from KGs and adaptively fuses them to generate high-quality embeddings for link prediction. Specifically, we introduce a MFFE approach designed to enhance predictive accuracy by effectively integrating diverse feature representations. Our framework comprises four core modules: (a) Relation-Conditioned Neighborhood Aggregator (RCNA), (b) Dynamic Relation Modulator (DRM), © Entity-Relation Attention Module (ERAM), and (d) Adaptive Feature Fusion Module(AFFM).
在本节中，我们首先明确了在知识图谱（KG）中解决链接预测问题的任务定义，随后提出了一种基于多特征融合嵌入的方法。图 1 展示了我们提出的 MultiFeature Fusion Embedding（MFFE）框架的整体架构，该框架通过从 KG 中提取多样化特征并进行自适应融合，生成高质量的嵌入向量用于链接预测。具体而言，我们提出了一种 MFFE 方法，旨在通过有效整合多样化特征表示来提升预测准确性。我们的框架包含四个核心模块：(a) 关系条件邻域聚合器（RCNA），(b) 动态关系调制器（DRM），(c) 实体关系注意力模块（ERAM），以及(d) 自适应特征融合模块（AFFM）。

A knowledge graph ( KG ) can be represented as $\mathcal{G}=$$\mathcal{G}=$G=\mathcal{G}= $(\mathcal{E},\mathcal{R},\mathcal{T})$$(\mathcal{E},\mathcal{R},\mathcal{T})$(E,R,T)(\mathcal{E}, \mathcal{R}, \mathcal{T}), where: $\mathcal{E}$$\mathcal{E}$E\mathcal{E} is the set of entities in the graph, $\mathcal{R}$$\mathcal{R}$R\mathcal{R} is the set of relations, $\mathcal{T}$$\mathcal{T}$T\mathcal{T} is the set of triples, where each triple consists of a head entity, a relation, and a tail entity.
知识图谱（KG）可以表示为 $\mathcal{G}=$$\mathcal{G}=$G=\mathcal{G}= $(\mathcal{E},\mathcal{R},\mathcal{T})$$(\mathcal{E},\mathcal{R},\mathcal{T})$(E,R,T)(\mathcal{E}, \mathcal{R}, \mathcal{T}) ，其中： $\mathcal{E}$$\mathcal{E}$E\mathcal{E} 是图中的实体集合， $\mathcal{R}$$\mathcal{R}$R\mathcal{R} 是关系集合， $\mathcal{T}$$\mathcal{T}$T\mathcal{T} 是三元组集合，每个三元组由一个头实体、一个关系和一个尾实体组成。

Each triple in $\mathcal{T}$$\mathcal{T}$T\mathcal{T} is represented as ( $e_h,e_r,e_t$$e_h,e_r,e_t$e_(h),e_(r),e_(t)e_{h}, e_{r}, e_{t} ), where $e_h,e_t\in \mathcal{E}$$e_h,e_t\in \mathcal{E}$e_(h),e_(t)inEe_{h}, e_{t} \in \mathcal{E} are the head and tail entities, and $e_r\in \mathcal{R}$$e_r\in \mathcal{R}$e_(r)inRe_{r} \in \mathcal{R} is the relation between the entities. In the context of link prediction, the task is to predict the missing entity in a given triple ( $e_h,e_r,?$$e_h,e_r,?$e_(h),e_(r),?e_{h}, e_{r}, ? ) or (?, $e_r,e_t$$e_r,e_t$e_(r),e_(t)e_{r}, e_{t} ), where either the head entity $e_h$$e_h$e_(h)e_{h} or the tail entity $e_t$$e_t$e_(t)e_{t} is missing.
每个三元组在 $\mathcal{T}$$\mathcal{T}$T\mathcal{T} 中表示为( $e_h,e_r,e_t$$e_h,e_r,e_t$e_(h),e_(r),e_(t)e_{h}, e_{r}, e_{t} )，其中 $e_h,e_t\in \mathcal{E}$$e_h,e_t\in \mathcal{E}$e_(h),e_(t)inEe_{h}, e_{t} \in \mathcal{E} 是头实体和尾实体， $e_r\in \mathcal{R}$$e_r\in \mathcal{R}$e_(r)inRe_{r} \in \mathcal{R} 是实体之间的关系。在链接预测的上下文中，任务是预测给定三元组（ $e_h,e_r,?$$e_h,e_r,?$e_(h),e_(r),?e_{h}, e_{r}, ? ）或（?, $e_r,e_t$$e_r,e_t$e_(r),e_(t)e_{r}, e_{t} ）中缺失的实体，其中头实体 $e_h$$e_h$e_(h)e_{h} 或尾实体 $e_t$$e_t$e_(t)e_{t} 缺失。

Formally, the link prediction problem is defined as follows: given a triple ( $e_h,e_r$$e_h,e_r$e_(h),e_(r)e_{h}, e_{r},?), the goal is to predict the tail entity $e_t$$e_t$e_(t)e_{t}, or given ( $?,e_r,e_t$$?,e_r,e_t$?,e_(r),e_(t)?, e_{r}, e_{t} ), the goal is to predict the head entity $e_h$$e_h$e_(h)e_{h}. This task can be represented as:
从形式上讲，链接预测问题定义如下：给定一个三元组 ( $e_h,e_r$$e_h,e_r$e_(h),e_(r)e_{h}, e_{r} ,?)，目标是预测尾部实体 $e_t$$e_t$e_(t)e_{t} ，或者给定 ( $?,e_r,e_t$$?,e_r,e_t$?,e_(r),e_(t)?, e_{r}, e_{t} )，目标是预测头部实体 $e_h$$e_h$e_(h)e_{h} 。该任务可表示为：

Fig.2. Overview of the feature extraction components in the proposed multi-feature fusion knowledge graph embedding framework. It includes: (a) Relation-aware Convolutional Network Aggregator (RCNA), (b) Dynamic Representation Module (DRM), and © Entity-Relation Alignment Mechanism (ERAM).
图 2. 提出的基于多特征融合知识图谱嵌入框架的特征提取组件概述。该框架包括：(a) 关系感知卷积网络聚合器（RCNA），(b) 动态表示模块（DRM），以及 © 实体关系对齐机制（ERAM）。

To effectively capture relationship-specific topological patterns in KGs, this section introduces the RCNA. This module aggregates neighbor features in a structure-sensitive way by dynamically parameterizing relationship embeddings as a convolution kernel during chain reasoning. The module consists of two core stages:
为了有效捕获知识图谱（KGs）中关系特异性的拓扑模式，本节引入了关系链分析（RCNA）模块。该模块通过在链推理过程中动态参数化关系嵌入作为卷积核，以结构敏感的方式聚合邻居特征。该模块包含两个核心阶段：
(1)Neighbor Subgraph Tensor Construction
(1)邻域子图张量构造

For a target head entity $h\in \mathcal{E}$$h\in \mathcal{E}$h inEh \in \mathcal{E}, a set of $K$$K$KK associated triples within its 1 -hop neighborhood, denoted as ${\left\{(h,r_i,t_i)\right\}}_{i=1}^K$${\left\{\left(h,r_i,t_i\right)\right\}}_{i=1}^K${(h,r_(i),t_(i))}_(i=1)^(K)\left\{\left(h, r_{i}, t_{i}\right)\right\}_{i=1}^{K} are systematically aggregated to construct a local subgraph, where $K$$K$KK is configured to be 32 by default. Following the embedding of each element into a $d$$d$dd-dimensional vector,
对于目标头实体 $h\in \mathcal{E}$$h\in \mathcal{E}$h inEh \in \mathcal{E} ，在其 1 跳邻域内与之关联的一组三元组 $K$$K$KK ，记为 ${\left\{(h,r_i,t_i)\right\}}_{i=1}^K$${\left\{\left(h,r_i,t_i\right)\right\}}_{i=1}^K${(h,r_(i),t_(i))}_(i=1)^(K)\left\{\left(h, r_{i}, t_{i}\right)\right\}_{i=1}^{K} ，系统性地聚合这些三元组以构建局部子图，其中 $K$$K$KK 默认配置为 32。随后，将每个元素嵌入到一个 $d$$d$dd 维向量中，
a three-dimensional structural tensor is subsequently constructed:
随后构建了一个三维结构张量：
${\mathbf{N}}_{\text{subgraph}}=\left[\mathrm{Pad}\left({\{\mathbf{h}\oplus {\mathbf{r}}_i\oplus {\mathbf{t}}_i\}}_{i=1}^{32}\right)\right]\in {\mathbb{R}}^{32\times 3\times d}$${\mathbf{N}}_{\text{subgraph }}=\left[\mathrm{Pad}\left({\left\{\mathbf{h}\oplus {\mathbf{r}}_i\oplus {\mathbf{t}}_i\right\}}_{i=1}^{32}\right)\right]\in {\mathbb{R}}^{32\times 3\times d}$N_("subgraph ")=[Pad({ho+r_(i)o+t_(i)}_(i=1)^(32))]inR^(32 xx3xx d)\mathbf{N}_{\text {subgraph }}=\left[\operatorname{Pad}\left(\left\{\mathbf{h} \oplus \mathbf{r}_{i} \oplus \mathbf{t}_{i}\right\}_{i=1}^{32}\right)\right] \in \mathbb{R}^{32 \times 3 \times d}
Here, $\oplus$$\oplus$o+\oplus denotes the concatenation operation, and the padding operation Pad is applied to ensure a fixed neighborhood size of 32 . When $K<32$$K<32$K < 32K<32, the remaining positions are padded with zero vectors, thereby ensuring that each entity is associated with a fixed-size tensor of dimensions. In contrast, if an entity has more than 32 neighbors, only the first 32 triples are retained to maintain consistency in tensor dimensions. The tensor preserves the topological ordering of the local subgraph along the second dimension, which corresponds to the neighbor index axis.
在此， $\oplus$$\oplus$o+\oplus 表示连接操作，填充操作 Pad 被应用以确保邻域大小固定为 32。当 $K<32$$K<32$K < 32K<32 时，剩余位置用零向量填充，从而确保每个实体与固定维度的张量相关联。相反，如果一个实体有超过 32 个邻居，则仅保留前 32 个三元组以保持张量维度的一致性。张量保留了局部子图在第二维度上的拓扑顺序，该维度对应于邻居索引轴。
(2)Relation-Conditioned Convolution Transformation The relationship embedding $\mathbf{r}\in {\mathbb{R}}^d$$\mathbf{r}\in {\mathbb{R}}^d$rinR^(d)\mathbf{r} \in \mathbb{R}^{d} from the current chain reasoning stage is projected into a dynamic convolution kernel via a fully connected layer. Specifically, the MLP projects $\mathbf{r}$$\mathbf{r}$r\mathbf{r} into a one-dimensional vector of size $32\times 3\times 3=288$$32\times 3\times 3=288$32 xx3xx3=28832 \times 3 \times 3=288, which is then reshaped into a three-dimensional tensor:
(2)关系条件卷积变换当前链推理阶段的关系嵌入 $\mathbf{r}\in {\mathbb{R}}^d$$\mathbf{r}\in {\mathbb{R}}^d$rinR^(d)\mathbf{r} \in \mathbb{R}^{d} 通过全连接层投影到动态卷积核中。具体而言，MLP 将 $\mathbf{r}$$\mathbf{r}$r\mathbf{r} 投影为大小为 $32\times 3\times 3=288$$32\times 3\times 3=288$32 xx3xx3=28832 \times 3 \times 3=288 的一维向量，随后将其重塑为三维张量：
${\mathcal{K}}_r=\mathrm{Reshape}(\mathrm{MLP}(\mathbf{r}))\in {\mathbb{R}}^{32\times 3\times 3}$${\mathcal{K}}_r=\mathrm{Reshape}(\mathrm{MLP}(\mathbf{r}))\in {\mathbb{R}}^{32\times 3\times 3}$K_(r)=Reshape(MLP(r))inR^(32 xx3xx3)\mathcal{K}_{r}=\operatorname{Reshape}(\operatorname{MLP}(\mathbf{r})) \in \mathbb{R}^{32 \times 3 \times 3}
The convolution kernel operates over a $3\times 3$$3\times 3$3xx33 \times 3 spatial window to model structural interactions within each triple, and utilizes 32 feature channels to learn discriminative representations for distinct neighboring entities. The convolution operation is formally defined as:
卷积核在 $3\times 3$$3\times 3$3xx33 \times 3 空间窗口上进行操作，以建模每个三元组内的结构交互，并利用 32 个特征通道学习区分相邻实体的特征表示。卷积操作的正式定义为：
${\mathbf{h}}_{\text{neigh}}=\sigma (\sum _{i=1}^{32}\sum _{j=1}^3\sum _{k=1}^3{\mathcal{K}}_r^{(i,j,k)}\ast {\mathbf{N}}_{\text{subgraph}}{}^{(i,j,k)})\in {\mathbb{R}}^d$${\mathbf{h}}_{\text{neigh }}=\sigma \left(\sum _{i=1}^{32} \sum _{j=1}^3 \sum _{k=1}^3 {\mathcal{K}}_r^{(i,j,k)}\ast {\mathbf{N}}_{\text{subgraph }}{}^{(i,j,k)}\right)\in {\mathbb{R}}^d$h_("neigh ")=sigma(sum_(i=1)^(32)sum_(j=1)^(3)sum_(k=1)^(3)K_(r)^((i,j,k))**N_("subgraph ")^((i,j,k)))inR^(d)\mathbf{h}_{\text {neigh }}=\sigma\left(\sum_{i=1}^{32} \sum_{j=1}^{3} \sum_{k=1}^{3} \mathcal{K}_{r}^{(i, j, k)} * \mathbf{N}_{\text {subgraph }}{ }^{(i, j, k)}\right) \in \mathbb{R}^{d}
where $\ast$$\ast$*** denotes the valid convolution operation on tensor slices, and $\sigma$$\sigma$sigma\sigma is the ReLU activation function. The final output neighbor features ${\mathbf{h}}_{\text{neigh}}$${\mathbf{h}}_{\text{neigh }}$h_("neigh ")\mathbf{h}_{\text {neigh }} will serve as relation-specific contextual representations for subsequent decoding.
其中， $\ast$$\ast$*** 表示张量切片上的有效卷积运算， $\sigma$$\sigma$sigma\sigma 为 ReLU 激活函数。最终输出的邻域特征 ${\mathbf{h}}_{\text{neigh}}$${\mathbf{h}}_{\text{neigh }}$h_("neigh ")\mathbf{h}_{\text {neigh }} 将作为后续解码的关联特异性上下文表示。

Traditional KGE models, such as RESCAL, represent each relation with a fixed matrix. Nevertheless, this static representation framework is insufficient for fostering the model’s adaptation to the relational impact on entities, particularly in accordance with the specific requirements of the task or the subtleties of the context. To overcome this limitation, we propose the DRM, which allows for the dynamic adjustment of relation embeddings.
传统 KGE 模型（如 RESCAL）通过固定矩阵表示每种关系。然而，这种静态表示框架无法有效支持模型适应关系对实体的影响，尤其难以满足任务特定需求或处理上下文细微差别。为克服这一局限，我们提出动态关系模型（DRM），该模型支持关系嵌入的动态调整。

In this module, each relation is initially represented by an embedding vector $\mathbf{r}$$\mathbf{r}$r\mathbf{r}. Instead of using a fixed matrix for each relation, we generate a dynamic relation matrix ${\mathbf{W}}_r$${\mathbf{W}}_r$W_(r)\mathbf{W}_{r} using a feed-forward neural network $f\theta$$f\theta$f thetaf \theta, which takes the relation embedding $\mathbf{r}$$\mathbf{r}$r\mathbf{r} as input.
在本模块中，每个关系最初由一个嵌入向量 $\mathbf{r}$$\mathbf{r}$r\mathbf{r} 表示。与为每个关系使用固定矩阵不同，我们通过一个前馈神经网络 $f\theta$$f\theta$f thetaf \theta 生成动态关系矩阵 ${\mathbf{W}}_r$${\mathbf{W}}_r$W_(r)\mathbf{W}_{r} ，该网络以关系嵌入 $\mathbf{r}$$\mathbf{r}$r\mathbf{r} 作为输入。
$\mathbf{W}r=f\theta (\mathbf{r})=\mathrm{ReLU}(\mathbf{r}{\mathbf{U}}_1+{\mathbf{b}}_1){\mathbf{U}}_2+{\mathbf{b}}_2$$\mathbf{W}r=f\theta (\mathbf{r})=\mathrm{ReLU}\left(\mathbf{r}{\mathbf{U}}_1+{\mathbf{b}}_1\right){\mathbf{U}}_2+{\mathbf{b}}_2$Wr=f theta(r)=ReLU(rU_(1)+b_(1))U_(2)+b_(2)\mathbf{W} r=f \theta(\mathbf{r})=\operatorname{ReLU}\left(\mathbf{r} \mathbf{U}_{1}+\mathbf{b}_{1}\right) \mathbf{U}_{2}+\mathbf{b}_{2}
where ${\mathbf{U}}_1\in {\mathbb{R}}^{d\times d}$${\mathbf{U}}_1\in {\mathbb{R}}^{d\times d}$U_(1)inR^(d xx d)\mathbf{U}_{1} \in \mathbb{R}^{d \times d} and ${\mathbf{U}}_2\in {\mathbb{R}}^{d\times d^2}$${\mathbf{U}}_2\in {\mathbb{R}}^{d\times d^2}$U_(2)inR^(d xxd^(2))\mathbf{U}_{2} \in \mathbb{R}^{d \times d^{2}} are learnable transformation matrices, ${\mathbf{b}}_1\in {\mathbb{R}}^{d_h}$${\mathbf{b}}_1\in {\mathbb{R}}^{d_h}$b_(1)inR^(d_(h))\mathbf{b}_{1} \in \mathbb{R}^{d_{h}} and ${\mathbf{b}}_2\in {\mathbb{R}}^{d^2}$${\mathbf{b}}_2\in {\mathbb{R}}^{d^2}$b_(2)inR^(d^(2))\mathbf{b}_{2} \in \mathbb{R}^{d^{2}} are bias vectors.
其中 ${\mathbf{U}}_1\in {\mathbb{R}}^{d\times d}$${\mathbf{U}}_1\in {\mathbb{R}}^{d\times d}$U_(1)inR^(d xx d)\mathbf{U}_{1} \in \mathbb{R}^{d \times d} 和 ${\mathbf{U}}_2\in {\mathbb{R}}^{d\times d^2}$${\mathbf{U}}_2\in {\mathbb{R}}^{d\times d^2}$U_(2)inR^(d xxd^(2))\mathbf{U}_{2} \in \mathbb{R}^{d \times d^{2}} 是可学习的变换矩阵， ${\mathbf{b}}_1\in {\mathbb{R}}^{d_h}$${\mathbf{b}}_1\in {\mathbb{R}}^{d_h}$b_(1)inR^(d_(h))\mathbf{b}_{1} \in \mathbb{R}^{d_{h}} 和 ${\mathbf{b}}_2\in {\mathbb{R}}^{d^2}$${\mathbf{b}}_2\in {\mathbb{R}}^{d^2}$b_(2)inR^(d^(2))\mathbf{b}_{2} \in \mathbb{R}^{d^{2}} 是偏置向量。

The output is then reshaped to form the dynamic relation matrix ${\mathbf{W}}_r\in {\mathbb{R}}^{d\times d}$${\mathbf{W}}_r\in {\mathbb{R}}^{d\times d}$W_(r)inR^(d xx d)\mathbf{W}_{r} \in \mathbb{R}^{d \times d}.
输出随后被重新 shaping 以形成动态关系矩阵 ${\mathbf{W}}_r\in {\mathbb{R}}^{d\times d}$${\mathbf{W}}_r\in {\mathbb{R}}^{d\times d}$W_(r)inR^(d xx d)\mathbf{W}_{r} \in \mathbb{R}^{d \times d} 。

The relation-aware feature extraction is performed by multiplying the entity embedding $\mathbf{h}$$\mathbf{h}$h\mathbf{h} with the dynamic relation matrix ${\mathbf{W}}_r$${\mathbf{W}}_r$W_(r)\mathbf{W}_{r}, as follows:
关系感知特征提取通过将实体嵌入 $\mathbf{h}$$\mathbf{h}$h\mathbf{h} 与动态关系矩阵 ${\mathbf{W}}_r$${\mathbf{W}}_r$W_(r)\mathbf{W}_{r} 相乘来实现，具体如下：
${\mathbf{h}}_{\text{rel}}=\mathbf{h}\cdot {\mathbf{W}}_r$${\mathbf{h}}_{\text{rel }}=\mathbf{h}\cdot {\mathbf{W}}_r$h_("rel ")=h*W_(r)\mathbf{h}_{\text {rel }}=\mathbf{h} \cdot \mathbf{W}_{r}
where $\mathbf{h}\in {\mathbb{R}}^{1\times d}$$\mathbf{h}\in {\mathbb{R}}^{1\times d}$hinR^(1xx d)\mathbf{h} \in \mathbb{R}^{1 \times d} denotes the original entity embedding vector, ${\mathbf{W}}_r\in {\mathbb{R}}^{d\times d}$${\mathbf{W}}_r\in {\mathbb{R}}^{d\times d}$W_(r)inR^(d xx d)\mathbf{W}_{r} \in \mathbb{R}^{d \times d} is the dynamic transformation matrix generated for relation $r$$r$rr, and ${\mathbf{h}}_{\text{rel}}\in {\mathbb{R}}^{1\times d}$${\mathbf{h}}_{\text{rel }}\in {\mathbb{R}}^{1\times d}$h_("rel ")inR^(1xx d)\mathbf{h}_{\text {rel }} \in \mathbb{R}^{1 \times d} represents the resulting relation-aware entity representation.
其中， $\mathbf{h}\in {\mathbb{R}}^{1\times d}$$\mathbf{h}\in {\mathbb{R}}^{1\times d}$hinR^(1xx d)\mathbf{h} \in \mathbb{R}^{1 \times d} 表示原始实体嵌入向量， ${\mathbf{W}}_r\in {\mathbb{R}}^{d\times d}$${\mathbf{W}}_r\in {\mathbb{R}}^{d\times d}$W_(r)inR^(d xx d)\mathbf{W}_{r} \in \mathbb{R}^{d \times d} 是为关系 $r$$r$rr 生成的动态变换矩阵，而 ${\mathbf{h}}_{\text{rel}}\in {\mathbb{R}}^{1\times d}$${\mathbf{h}}_{\text{rel }}\in {\mathbb{R}}^{1\times d}$h_("rel ")inR^(1xx d)\mathbf{h}_{\text {rel }} \in \mathbb{R}^{1 \times d} 表示最终获得的关系感知实体表示。

This formulation allows each relation to have its own specific effect on the entity, providing a richer and more expressive representation of the entity-relation interactions. By dynamically adjusting the relation matrix, the model is capable of capturing the diverse and complex nature of the relations in the knowledge graph.
该方法使每个关系都能对实体产生其特有的影响，从而对实体与关系之间的交互关系提供更丰富、更具表达力的表示。通过动态调整关系矩阵，该模型能够捕捉知识图谱中关系的多样性和复杂性。

We introduce the ERAM based on multi-head attention to enhance the interaction between entities and relations. Specifically, entity embeddings, relation embeddings, and the dot product of entities and relations are designated as query vectors, key vectors, and value vectors, respectively. The attention weights are dynamically computed through the multi-head attention mechanism, which reflects the varying importance of different relations to the entity representation. These weights highlight key relations and diminish redundant information. The weighted aggregation of these attention weights with the entity-relation representations results in the final feature representation, capturing the refined interaction between the entity and relation.
我们提出了一种基于多头注意力的 ERAM 模型，以增强实体与关系之间的交互作用。具体而言，实体嵌入、关系嵌入以及实体与关系的点积分别被指定为查询向量、关键向量和值向量。通过多头注意力机制动态计算注意力权重，反映不同关系对实体表示的不同重要性。这些权重突出了关键关系并削弱了冗余信息。通过将这些注意力权重与实体-关系表示进行加权聚合，最终得到特征表示，从而捕捉到实体与关系之间精细化的交互关系。

Formally, for each triple ( $s,r,t$$s,r,t$s,r,ts, r, t ), we assign an embedding vector ${\mathbf{e}}_s\in {\mathbb{R}}^d$${\mathbf{e}}_s\in {\mathbb{R}}^d$e_(s)inR^(d)\mathbf{e}_{s} \in \mathbb{R}^{d} to the head entity $s$$s$ss, and ${\mathbf{e}}_r\in {\mathbb{R}}^d$${\mathbf{e}}_r\in {\mathbb{R}}^d$e_(r)inR^(d)\mathbf{e}_{r} \in \mathbb{R}^{d} to the relation $r$$r$rr. These embeddings are used to form query, key, and value vectors for the multi-head attention mechanism. The query matrix ${\mathbf{Q}}_s$${\mathbf{Q}}_s$Q_(s)\mathbf{Q}_{s} is derived from the entity embedding ${\mathbf{e}}_s$${\mathbf{e}}_s$e_(s)\mathbf{e}_{s}, while the key matrix ${\mathbf{K}}_r$${\mathbf{K}}_r$K_(r)\mathbf{K}_{r} comes from the relation embedding ${\mathbf{e}}_r$${\mathbf{e}}_r$e_(r)\mathbf{e}_{r}, and the value matrix ${\mathbf{V}}_{s}r$${\mathbf{V}}_{s}r$V_(s)r\mathbf{V}_{s} r is the dot product of the entity and relation embeddings. The attention matrix ${\mathbf{A}}_{sr}$${\mathbf{A}}_{sr}$A_(sr)\mathbf{A}_{s r}, which measures the influence of the relation $r$$r$rr on the entity $s$$s$ss, is calculated as:
正式地，对于每个三元组 ( $s,r,t$$s,r,t$s,r,ts, r, t )，我们将嵌入向量 ${\mathbf{e}}_s\in {\mathbb{R}}^d$${\mathbf{e}}_s\in {\mathbb{R}}^d$e_(s)inR^(d)\mathbf{e}_{s} \in \mathbb{R}^{d} 映射到头实体 $s$$s$ss ，并将 ${\mathbf{e}}_r\in {\mathbb{R}}^d$${\mathbf{e}}_r\in {\mathbb{R}}^d$e_(r)inR^(d)\mathbf{e}_{r} \in \mathbb{R}^{d} 映射到关系 $r$$r$rr 。这些嵌入向量用于构造多头注意力机制中的查询向量、键向量和值向量。查询矩阵 ${\mathbf{Q}}_s$${\mathbf{Q}}_s$Q_(s)\mathbf{Q}_{s} 由实体嵌入 ${\mathbf{e}}_s$${\mathbf{e}}_s$e_(s)\mathbf{e}_{s} 衍生而来，而键矩阵 ${\mathbf{K}}_r$${\mathbf{K}}_r$K_(r)\mathbf{K}_{r} 来自关系嵌入 ${\mathbf{e}}_r$${\mathbf{e}}_r$e_(r)\mathbf{e}_{r} ，值矩阵 ${\mathbf{V}}_{s}r$${\mathbf{V}}_{s}r$V_(s)r\mathbf{V}_{s} r 则是实体嵌入与关系嵌入的点积。注意力矩阵 ${\mathbf{A}}_{sr}$${\mathbf{A}}_{sr}$A_(sr)\mathbf{A}_{s r} ，用于衡量关系 $r$$r$rr 对实体 $s$$s$ss 的影响，其计算方式为：
${\mathbf{A}}_{sr}=\mathrm{softmax}\left(\frac{{\mathbf{e}}_s{\mathbf{e}}_r^T}{\sqrt{d}}\right)$${\mathbf{A}}_{sr}=\mathrm{softmax}\left(\frac{{\mathbf{e}}_s{\mathbf{e}}_r^T}{\sqrt{d}}\right)$A_(sr)=softmax((e_(s)e_(r)^(T))/(sqrtd))\mathbf{A}_{s r}=\operatorname{softmax}\left(\frac{\mathbf{e}_{s} \mathbf{e}_{r}^{T}}{\sqrt{d}}\right)
where ${\mathbf{e}}_s,{\mathbf{e}}_r\in {\mathbb{R}}^{1\times d}$${\mathbf{e}}_s,{\mathbf{e}}_r\in {\mathbb{R}}^{1\times d}$e_(s),e_(r)inR^(1xx d)\mathbf{e}_{s}, \mathbf{e}_{r} \in \mathbb{R}^{1 \times d} denote the embedding vectors of the head entity and relation, respectively, and $d$$d$dd is the embedding dimension.
其中， ${\mathbf{e}}_s,{\mathbf{e}}_r\in {\mathbb{R}}^{1\times d}$${\mathbf{e}}_s,{\mathbf{e}}_r\in {\mathbb{R}}^{1\times d}$e_(s),e_(r)inR^(1xx d)\mathbf{e}_{s}, \mathbf{e}_{r} \in \mathbb{R}^{1 \times d} 分别表示头实体和关系的嵌入向量，而 $d$$d$dd 表示嵌入维度。

Subsequently, we compute the weighted feature representation ${\mathbf{h}}_{\text{int}}$${\mathbf{h}}_{\text{int }}$h_("int ")\mathbf{h}_{\text {int }} by aggregating the value vectors ${\mathbf{V}}_r$${\mathbf{V}}_r$V_(r)\mathbf{V}_{r} using the attention matrix ${\mathbf{A}}_{sr}$${\mathbf{A}}_{sr}$A_(sr)\mathbf{A}_{s r} :
随后，我们通过使用注意力矩阵 ${\mathbf{A}}_{sr}$${\mathbf{A}}_{sr}$A_(sr)\mathbf{A}_{s r} 聚合值向量 ${\mathbf{V}}_r$${\mathbf{V}}_r$V_(r)\mathbf{V}_{r} ，计算加权特征表示 ${\mathbf{h}}_{\text{int}}$${\mathbf{h}}_{\text{int }}$h_("int ")\mathbf{h}_{\text {int }} ：
${\mathbf{h}}_{int}={\mathbf{A}}_{sr}({\mathbf{e}}_s\odot {\mathbf{e}}_r)$${\mathbf{h}}_{int}={\mathbf{A}}_{sr}\left({\mathbf{e}}_s\odot {\mathbf{e}}_r\right)$h_(int)=A_(sr)(e_(s)o.e_(r))\mathbf{h}_{i n t}=\mathbf{A}_{s r}\left(\mathbf{e}_{s} \odot \mathbf{e}_{r}\right)
The output ${\mathbf{h}}_{\text{int}}$${\mathbf{h}}_{\text{int }}$h_("int ")\mathbf{h}_{\text {int }} represents the refined feature of the entity after incorporating the relation’s influence.
输出 ${\mathbf{h}}_{\text{int}}$${\mathbf{h}}_{\text{int }}$h_("int ")\mathbf{h}_{\text {int }} 表示在考虑关系影响后，实体的精炼特征。

To further enhance the expressiveness of the model, we apply multi-head attention, where multiple sets of query, key, and value projections are used to independently compute attention for each head. The outputs of all heads are concatenated and linearly transformed. Mathematically, the multi-head attention mechanism can be defined as:
为了进一步提升模型的表达能力，我们引入了多头注意力机制，其中使用多组查询、关键和值投影来独立计算每个头部的注意力。所有头部的输出被拼接后再进行线性变换。数学上，多头注意力机制可定义为：

Mathematically, the multi-head attention mechanism can be defined as:
从数学上讲，多头注意力机制可以定义为：
$\mathrm{MultiHead}({\mathbf{e}}_s,{\mathbf{e}}_r)=[{\mathbf{h}}_{{\text{int}}_1},{\mathbf{h}}_{{\text{int}}_2},\dots ,{\mathbf{h}}_{{\text{int}}_i}]{\mathbf{W}}_O$$\mathrm{MultiHead}\left({\mathbf{e}}_s,{\mathbf{e}}_r\right)=\left[{\mathbf{h}}_{{\text{int }}_1},{\mathbf{h}}_{{\text{int }}_2},\dots ,{\mathbf{h}}_{{\text{int }}_i}\right]{\mathbf{W}}_O$MultiHead(e_(s),e_(r))=[h_("int "_(1)),h_("int "_(2)),dots,h_("int "_(i))]W_(O)\operatorname{MultiHead}\left(\mathbf{e}_{s}, \mathbf{e}_{r}\right)=\left[\mathbf{h}_{\text {int }_{1}}, \mathbf{h}_{\text {int }_{2}}, \ldots, \mathbf{h}_{\text {int }_{i}}\right] \mathbf{W}_{O}
where ${\mathbf{h}}_{{\text{int}}_i}$${\mathbf{h}}_{{\text{int }}_i}$h_("int "_(i))\mathbf{h}_{\text {int }_{i}} denotes the output of the $i$$i$ii-th attention head, and ${\mathbf{W}}_O\in {\mathbb{R}}^{h\cdot d\times d}$${\mathbf{W}}_O\in {\mathbb{R}}^{h\cdot d\times d}$W_(O)inR^(h*d xx d)\mathbf{W}_{O} \in \mathbb{R}^{h \cdot d \times d} is the output projection matrix.
其中， ${\mathbf{h}}_{{\text{int}}_i}$${\mathbf{h}}_{{\text{int }}_i}$h_("int "_(i))\mathbf{h}_{\text {int }_{i}} 表示第 $i$$i$ii 个注意力头部的输出，而 ${\mathbf{W}}_O\in {\mathbb{R}}^{h\cdot d\times d}$${\mathbf{W}}_O\in {\mathbb{R}}^{h\cdot d\times d}$W_(O)inR^(h*d xx d)\mathbf{W}_{O} \in \mathbb{R}^{h \cdot d \times d} 是输出投影矩阵。

The ERAM effectively captures the intricate interactions between entities and relations by dynamically adjusting the contribution of different relations to the entity’s representation. The multi-head attention mechanism allows the model to learn from multiple subspaces simultaneously, enhancing its ability to represent complex relational information in KGs.
ERAM 通过动态调整不同关系对实体表示的贡献，有效捕获实体与关系之间的复杂交互。多头注意力机制使模型能够同时从多个子空间中学习，从而提升其在知识图谱中表示复杂关系信息的能力。

Fig.3. Illustration of the Adaptive Feature Fusion Module (AFFM)
图 3. 自适应特征融合模块（AFFM）的示意图

The AFFM is designed to combine the features extracted from the previous three modules: neighborhood features, relation-aware features, and entity-relation interaction features. This module aims to aggregate these features effectively and produce a comprehensive entity representation that captures both local and global graph information.
AFFM 旨在整合前三个模块中提取的特征：邻域特征、关系感知特征以及实体关系交互特征。该模块的目标是有效聚合这些特征，生成一个全面的实体表示，能够同时捕捉局部和全局图信息。

Let ${\mathbf{h}}_{\text{neigh}}\in {\mathbb{R}}^d,{\mathbf{h}}_{\text{rel}}\in {\mathbb{R}}^d$${\mathbf{h}}_{\text{neigh }}\in {\mathbb{R}}^d,{\mathbf{h}}_{\text{rel }}\in {\mathbb{R}}^d$h_("neigh ")inR^(d),h_("rel ")inR^(d)\mathbf{h}_{\text {neigh }} \in \mathbb{R}^{d}, \mathbf{h}_{\text {rel }} \in \mathbb{R}^{d}, and ${\mathbf{h}}_{\text{int}}\in {\mathbb{R}}^d$${\mathbf{h}}_{\text{int }}\in {\mathbb{R}}^d$h_("int ")inR^(d)\mathbf{h}_{\text {int }} \in \mathbb{R}^{d} denote three distinct feature vectors corresponding to neighborhood features, relation-aware features, and entity-relation interaction features, respectively. The module operates through a cascaded architecture comprising bidirectional gated fusion and dynamic weight aggregation, enabling adaptive combination of complementary information across features.
令 ${\mathbf{h}}_{\text{neigh}}\in {\mathbb{R}}^d,{\mathbf{h}}_{\text{rel}}\in {\mathbb{R}}^d$${\mathbf{h}}_{\text{neigh }}\in {\mathbb{R}}^d,{\mathbf{h}}_{\text{rel }}\in {\mathbb{R}}^d$h_("neigh ")inR^(d),h_("rel ")inR^(d)\mathbf{h}_{\text {neigh }} \in \mathbb{R}^{d}, \mathbf{h}_{\text {rel }} \in \mathbb{R}^{d} 、 ${\mathbf{h}}_{\text{int}}\in {\mathbb{R}}^d$${\mathbf{h}}_{\text{int }}\in {\mathbb{R}}^d$h_("int ")inR^(d)\mathbf{h}_{\text {int }} \in \mathbb{R}^{d} 分别表示对应于邻域特征、关系感知特征和实体关系交互特征的三个不同特征向量。该模块通过级联架构实现，包含双向门控融合与动态权重聚合，从而实现跨特征的互补信息自适应融合。

For each feature pair ( ${\mathbf{h}}_i,{\mathbf{h}}_j$${\mathbf{h}}_i,{\mathbf{h}}_j$h_(i),h_(j)\mathbf{h}_{i}, \mathbf{h}_{j} ), where $i,j\in \{$$i,j\in \{$i,j in{i, j \in\{ neigh, rel, int $\}$$\}$}\}, the fusion process begins by modeling cross-feature dependencies through a parameterized gating mechanism. A convolutional operator with kernel ${\mathbf{W}}_{ij}\in {\mathbb{R}}^{k\times 2d}$${\mathbf{W}}_{ij}\in {\mathbb{R}}^{k\times 2d}$W_(ij)inR^(k xx2d)\mathbf{W}_{i j} \in \mathbb{R}^{k \times 2 d} processes the concatenated input $[{\mathbf{h}}_i;{\mathbf{h}}_j]$$\left[{\mathbf{h}}_i;{\mathbf{h}}_j\right]$[h_(i);h_(j)]\left[\mathbf{h}_{i} ; \mathbf{h}_{j}\right], generating a spatial-channel attention map:
对于每个特征对 ( ${\mathbf{h}}_i,{\mathbf{h}}_j$${\mathbf{h}}_i,{\mathbf{h}}_j$h_(i),h_(j)\mathbf{h}_{i}, \mathbf{h}_{j} )，其中 $i,j\in \{$$i,j\in \{$i,j in{i, j \in\{ 与 $\}$$\}$}\} 邻接且具有相同关系和整数属性，融合过程首先通过参数化门控机制建模跨特征依赖关系。卷积操作符（卷积核为 ${\mathbf{W}}_{ij}\in {\mathbb{R}}^{k\times 2d}$${\mathbf{W}}_{ij}\in {\mathbb{R}}^{k\times 2d}$W_(ij)inR^(k xx2d)\mathbf{W}_{i j} \in \mathbb{R}^{k \times 2 d} ）处理拼接后的输入 $[{\mathbf{h}}_i;{\mathbf{h}}_j]$$\left[{\mathbf{h}}_i;{\mathbf{h}}_j\right]$[h_(i);h_(j)]\left[\mathbf{h}_{i} ; \mathbf{h}_{j}\right] ，生成空间-通道注意力图：
${\mathbf{S}}_{ij}=\sigma (\mathrm{Conv}([h_i;h_j],W_{ij})+b_{ij})$${\mathbf{S}}_{ij}=\sigma \left(\mathrm{Conv}\left(\left[h_i;h_j\right],W_{ij}\right)+b_{ij}\right)$S_(ij)=sigma(Conv([h_(i);h_(j)],W_(ij))+b_(ij))\mathbf{S}_{i j}=\sigma\left(\operatorname{Conv}\left(\left[h_{i} ; h_{j}\right], W_{i j}\right)+b_{i j}\right)
where $\sigma$$\sigma$sigma\sigma denotes the sigmoid function, and ${\mathbf{S}}_{ij}\in [0,1{]}^d$${\mathbf{S}}_{ij}\in [0,1{]}^d$S_(ij)in[0,1]^(d)\mathbf{S}_{i j} \in[0,1]^{d} dynamically quantifies the relevance of each feature dimension. The gated fusion output is then computed as:
其中 $\sigma$$\sigma$sigma\sigma 表示 sigmoid 函数，而 ${\mathbf{S}}_{ij}\in [0,1{]}^d$${\mathbf{S}}_{ij}\in [0,1{]}^d$S_(ij)in[0,1]^(d)\mathbf{S}_{i j} \in[0,1]^{d} 动态量化每个特征维度的相关性。门控融合输出随后计算为：
${\mathbf{F}}_{ij}={\mathbf{S}}_{ij}\odot {\mathbf{h}}_i+(\mathbf{1}-{\mathbf{S}}_{ij})\odot {\mathbf{h}}_j$${\mathbf{F}}_{ij}={\mathbf{S}}_{ij}\odot {\mathbf{h}}_i+\left(\mathbf{1}-{\mathbf{S}}_{ij}\right)\odot {\mathbf{h}}_j$F_(ij)=S_(ij)o.h_(i)+(1-S_(ij))o.h_(j)\mathbf{F}_{i j}=\mathbf{S}_{i j} \odot \mathbf{h}_{i}+\left(\mathbf{1}-\mathbf{S}_{i j}\right) \odot \mathbf{h}_{j},
where $\odot$$\odot$o.\odot represents element-wise multiplication. This formulation allows selective emphasis on discriminative patterns while suppressing redundant components between ${\mathbf{h}}_i$${\mathbf{h}}_i$h_(i)\mathbf{h}_{i} and ${\mathbf{h}}_j$${\mathbf{h}}_j$h_(j)\mathbf{h}_{j}.
其中 $\odot$$\odot$o.\odot 表示元素级乘法。这种表述方式允许有选择地强调区分性模式，同时抑制 ${\mathbf{h}}_i$${\mathbf{h}}_i$h_(i)\mathbf{h}_{i} 和 ${\mathbf{h}}_j$${\mathbf{h}}_j$h_(j)\mathbf{h}_{j} 之间的冗余成分。