PairNorm swMATH ID: 38087 Software Authors: Lingxiao Zhao, Leman Akoglu Description: PairNorm: Tackling Oversmoothing in GNNs. The performance of graph neural nets (GNNs) is known to gradually decrease with increasing number of layers. This decay is partly attributed to oversmoothing, where repeated graph convolutions eventually make node embeddings indistinguishable. We take a closer look at two different interpretations, aiming to quantify oversmoothing. Our main contribution is PairNorm, a novel normalization layer that is based on a careful analysis of the graph convolution operator, which prevents all node embeddings from becoming too similar. What is more, PairNorm is fast, easy to implement without any change to network architecture nor any additional parameters, and is broadly applicable to any GNN. Experiments on real-world graphs demonstrate that PairNorm makes deeper GCN, GAT, and SGC models more robust against oversmoothing, and significantly boosts performance for a new problem setting that benefits from deeper GNNs. Code is available at https://github.com/LingxiaoShawn/PairNorm Homepage: https://arxiv.org/abs/1909.12223 Source Code: https://github.com/LingxiaoShawn/PairNorm Keywords: Machine Learning; arXiv_cs.LG; arXiv_stat.ML Related Software: DropEdge; GraphSAGE; DeepGCNs; Eigen-GNN; PMTK; SIGN; GMNN; NetKit Cited in: 1 Document Cited by 2 Authors 1 Benson, Austin R. 1 Jia, Junteng Cited in 1 Serial 1 SIAM Journal on Mathematics of Data Science Cited in 4 Fields 1 Combinatorics (05-XX) 1 Statistics (62-XX) 1 Computer science (68-XX) 1 Game theory, economics, finance, and other social and behavioral sciences (91-XX) Citations by Year