This 4-credit PhD-level course covers state-of-the-art
research on data mining and machine learning with graphs. Topics include, but
are not limited to, vertex classification, graph clustering, link prediction and
analysis, graph distances, graph embedding and network representation learning,
deep learning on graphs, anomaly detection on graphs, graph summarization,
network inference, adversarial learning on networks, notions of fairness in
social networks, and generative AI methods on graphs.
Students are expected to have taken courses on or have
knowledge of the following:
This course does not have a designated textbook. The readings
are assigned in the syllabus (see below).
|
Date
|
Lecturer
|
Readings
|
|
Wed Sep 9
|
Tina Eliassi-Rad
|
Overview
|
|
Mon Sep 14
|
Tina Eliassi-Rad
|
Representations of Complex Systems
The Why, How, and
When of Representations for
Complex Systems
[optional]
Networks beyond
Pairwise Interactions: Structure and Dynamics
[optional]
The Physics of
Higher-order Interactions in Complex Systems
|
|
Wed Sep 16
|
Tina Eliassi-Rad
|
Homophily
Distribution of Node
Characteristics in Complex Networks
Combinatorial
Characterizations and Impossibilities for Higher-order Homophily
Higher-order Homophily
on Simplicial Complexes
[optional] Information
Access Equality on Generative Models of Complex Networks
[optional] Effects of Higher-order
Interactions and Homophily on Information Access Inequality
Axiomatic Approaches
Measuring Tie Strength in
Implicit Social Networks
|
|
Mon Sep 21
|
Tina Eliassi-Rad
|
Network Comparison and Graph Distances
A Guide to Selecting a
Network Similarity Method
Network
Comparison and the Within-ensemble Graph Distance
Non-backtracking
Cycles: Length Spectrum Theory and Graph Mining Applications
[reference] netrd:
A library for Network Reconstruction and Graph Distances
[optional] Optimal Transport for Network
Comparison: A Review with Machine Learning Applications
|
|
Wed Sep 23
|
Tina Eliassi-Rad
|
Role Discovery
It's Who You Know: Graph
Mining Using Recursive Structural Features
RolX:
Structural Role Extraction & Mining in Large Graphs
Guided learning for Role
Discovery (GLRD): Framework, Algorithms, and Applications
|
|
Mon Sep 28
&
Wed Sep 30
|
Tina Eliassi-Rad
|
Graph Representation Learning: Node Embedding
Laplacian
Eigenmaps for Dimensionality Reduction and Data Representation
node2vec:
Scalable Feature Learning for Networks
Structural
Deep Network Embedding
STABLE:
Identifying and Mitigating Instability in Embeddings of the Degenerate Core
[reference] Machine Learning on Graphs: A Model
and Comprehensive Taxonomy
[reference] Network Representation Learning:
From Preprocessing, Feature Extraction to Node Embedding
[optional] DeepWalk:
Online Learning of Social Representations
[optional] Bypassing Skip-Gram Negative
Sampling: Dimension Regularization as a More Efficient Alternative for Graph
Embeddings
[optional] REGE: A Method for
Incorporating Uncertainty in Graph Embeddings
[optional] Next Waves in Veridical Network
Embedding
|
|
Mon Oct 5
|
|
Low-rank Representations of Complex Networks
[paper 1] The Impossibility of
Low-rank Representations for Triangle-Rich Complex Networks
[paper 2] Node
Embeddings and Exact Low-rank Representations of Complex Networks
[paper 3] Link Prediction Using
Low-dimensional Node Embeddings: The Measurement Problem
[optional] Classic Graph Structural Features
Outperform Factorization-Based Graph Embedding Methods on Community Labeling
[optional] Network Embedding as Matrix
Factorization: Unifying DeepWalk, LINE, PTE, and
node2vec
|
|
Wed Oct 7
|
|
Graph Neural Networks I
[paper 1] Semi-Supervised
Classification with Graph Convolutional Networks
[paper 2] Graph
Attention Networks
[paper 3] Inductive
Representation Learning on Large Graphs
[reference] Graph Neural Networks: A Review of
Methods and Applications
[reference] A Comprehensive Survey on Graph
Neural Networks
[optional] Everything is Connected: Graph Neural
Networks
[optional] Geometric Deep
Learning: the Erlangen Programme of ML (ICLR 2021
keynote)
|
|
Mon Oct 12
|
No
class (US holiday)
|
|
Wed Oct 14
|
Class
project proposals are due at 11:59 PM Eastern.
|
Graph Neural Networks II
[paper 1] Hyperbolic
Graph Convolutional Neural Networks
[paper 2] Pitfalls
of Graph Neural Network Evaluation
[paper 3] Design
Space for Graph Neural Networks (GitHub page)
[optional] The Numerical Stability of Hyperbolic Representation Learning
|
|
Mon Oct 19
|
|
Graph Neural Networks III & Collective
Classification
[paper 1] Hierarchical
Graph Representation Learning with Differentiable Pooling
[paper 2] Collective
Classification in Network Data
[paper 3] Graph
Belief Propagation Networks
[optional] Cautious
Collective Classification
|
|
Wed Oct 21
|
|
Label Propagation on Graphs and Counting
[paper 1] Combining Label Propagation and
Simple Models Out-performs Graph Neural Networks
[paper 2] Masked Label
Prediction: Unified Message Passing Model for Semi-Supervised Classification
[paper 3] Can Graph Neural Networks Count
Substructures?
[optional] Message passing all the way up
|
|
Mon Oct 26
|
|
GNNs for Recommendation Systems
[paper 1] LightGCN: Simplifying and Powering Graph Convolution
Network for Recommendation
[paper 2] Neural Graph Collaborative
Filtering
[paper 3] Graph Convolutional Neural
Networks for Web-Scale Recommender Systems
[reference] Graph Neural Networks in Recommender
Systems: A Survey
[reference] A Survey of Graph Neural Networks for
Recommender Systems: Challenges, Methods, and Directions
|
|
Wed Oct 28
|
|
Hypergraphs and Higher-order Models with
applications to Graph ML for Optimization
[paper 1] Random Walks
on Hypergraphs with Edge-Dependent Vertex Weights
[paper 2] Assigning
Entities to Teams as a Hypergraph Discovery Problem
[paper 3] Distributed
constrained combinatorial optimization leveraging hypergraph neural networks
[reference] A Survey on Hypergraph Mining:
Patterns, Tools, and Generators
[optional] Hypergraph Neural Networks
[optional] Simplicial Attention Networks
|
|
Mon Nov 2
|
|
Oversmooting and Oversquashing
[paper 1] A
Survey on Oversmoothing in Graph Neural Networks
[paper 2] How
does over-squashing affect the power of GNNs?
[paper 3] Neural
Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs
|
|
Wed Nov 4
|
|
W-L Graph Kernels, Power of GNNs, Stability
[paper 1] Weisfeiler-Lehman Graph Kernels
[paper 2] How Powerful are Graph Neural
Networks?
[paper 3] Tree Mover’s Distance:
Bridging Graph Metrics and Stability of Graph Neural Networks
[optional] A Reduction
of a Graph to a Canonical Form and an Algebra arising during this Reduction
[reference] Theory of Graph Neural Networks:
Representation and Learning
|
|
Mon Nov 9
|
|
Invariance and Equivariance + 1
[paper 1] E(n)
Equivariant Graph Neural Networks
[paper 2] Invariant and Equivariant Graph
Networks
[paper
3] Uncertainty Quantification over
Graph with Conformalized Graph Neural Networks
|
|
Wed Nov 11
|
No class (US holiday)
|
|
|
Mon Nov 16
|
|
*-aware GNNs + 1
[paper 1] Position-aware Graph Neural
Networks
[paper 2] Identity-aware Graph Neural
Networks
[paper
3] PRODIGY: Enabling In-context
Learning Over Graphs
|
|
Wed Nov 18
|
|
Explainability in GNNs
[paper 1] GNNExplainer:
Generating Explanations for Graph Neural Networks
[paper 2] GraphFramEx:
Towards Systematic Evaluation of Explainability Methods for Graph Neural
Networks
[paper 3] Trustworthy Graph Neural
Networks: Aspects, Methods and Trends
[optional] FAIRGEN: Towards Fair Graph
Generation
[optional] Explainability in Graph Neural
Networks: A Taxonomic Survey
|
|
Mon Nov 23
|
|
Graph Transformers I
[paper 1] Graph
Transformer Networks
[paper 2] A Generalization of Transformer
Networks to Graphs
[paper 3] Transformers
are Graph Neural Networks
[reference] Graph Transformers: A Survey
|
|
Wed Nov 25
|
No class (US holiday)
|
|
|
Mon Nov 30
|
|
Graph Transformers II
[paper 1] Do Transformers Really Perform Bad
for Graph Representation?
[paper 2] Sign and Basis Invariant Networks
for Spectral Graph Representation Learning
[paper 3] Attending to Graph Transformers
[reference] Graph Transformers: A Survey
|
|
Wed Dec 2
|
|
ML on Heterogeneous Graphs + Deep
Generative Models for Graphs
[paper 1] Modeling Relational Data with
Graph Convolutional Networks
[paper 2] Heterogeneous Graph Transformer
[paper 3] GraphRNN: Generating Realistic Graphs with Deep
Auto-regressive Models
|
|
Mon Dec 7
|
|
Relational Deep Learning
[paper 1] Relational
Deep Learning - Graph Representation Learning on Relational Databases
[paper 2] Relational
Graph Transformer
[paper 3] RelGNN: Composite Message Passing for Relational Deep
Learning
[optional] Relational Transformer: Toward Zero-Shot
Foundation Models for Relational Data
[reference] RelBench: A
Benchmark for Deep Learning on Relational Databases
|
|
Wed Dec 9
|
Moderator: Samantha Dies
|
Knowledge Graphs
[paper 1] Translating
Embeddings for Modeling Multi-relational Data
[paper 2] Learning Entity
and Relation Embeddings for Knowledge Graph Completion
[paper 3] Embedding Entities and Relations
for Learning and Inference in Knowledge Bases
[optional] Complex Embeddings for Simple
Link Prediction
[optional] RotatE:
Knowledge Graph Embedding by Relational Rotation in Complex Space
|
|
Mon Dec 14
|
|
Recent Position Papers
[paper 1] Future Directions in the
Theory of Graph Machine Learning
[paper 2] Position:
Graph Learning Will Lose Relevance Due To Poor Benchmarks
[paper 3] Position: Graph
Condensation Needs a Reset—Move Beyond Full-dataset Training and
Model-Dependence
[paper 4] Position: Neural Approximation
Is Rarely Justified for Hard Combinatorial Problems
[paper 5] Why
We Must Rethink Empirical Research in Machine Learning
|
|
Wed Dec 16
|
Presentation of class projects
|
Reports
and slides for class projects are due at 12:00 PM (noon) Eastern.
|
|
Tue Dec 22
|
Faculty
Grade Deadline
|