A black and red text on a black background Description automatically generated

Fall 2023: NETS 7332 -- Machine Learning with Graphs (CRN 19900)

 

General Information

Time: Tuesdays & Fridays 1:35 – 3:15 PM Eastern

Place: 177 Huntington Ave, 2nd Floor Conference Room

Instructor: Tina Eliassi-Rad

Course website on Canvas: https://northeastern.instructure.com/courses/158581

Office hours: Available by appointment. Email eliassi [at] ccs [dot] neu [dot] edu
to setup an appointment; begin the subject line with [fa23 nets].

TA: Ayan Chatterjee. Available by appointment. Email chatterjee [dot] ay [at]
northeastern [dot] edu to setup an appointment; begin the subject line with [fa23 nets].

 

Overview

This 4-credit PhD-level course covers state-of-the-art research on mining and learning with graphs. Topics include, but are not limited to, vertex classification, graph clustering, link prediction and analysis, graph distances, graph embedding and network representation learning, deep learning on graphs, anomaly detection on graphs, graph summarization, network inference, adversarial learning on networks, and notions of fairness in social networks.

 

Prerequisites

Students are expected to have taken courses on or have knowledge of the following:

o   Calculus and linear algebra

o   Basic statistics, probability, machine learning, or data mining

o   Algorithms and programming skills (e.g., Python, Julia, C, Java, Matlab, or any programming language of their preference)

 

Textbooks

This course does not have a designated textbook. The readings are assigned in the syllabus (see below).

 

Here are some textbooks (all optional) on machine learning and data mining:

 

Resources

 

Grading

o   Class presentations (40%)

o   I will team up students into groups of two.

o   Each week has an assigned team. That team is responsible for presenting the readings for that week.

o   Besides the readings, each paper is likely to have additional materials on the Web. Examples include supplemental materials, video, code, data, etc. These are helpful for presentations and class projects.

o   Class project (50%)

o   Each team will choose (by Saturday October 7, 2023 at 11:59 PM Eastern) one of the papers in the syllabus to replicate.

o   In addition to replication, each team will propose extension(s) to the chosen paper and implement those extension(s).

o   Each team will write a report (maximum 6 pages) detailing what was learned. Use the style files at https://paperswithcode.com/static/rc2020/ML-Reproducibility-Challenge-2020-Template.zip.

o   Reports are due on Saturday December 9, 2023 at 11:59 PM Eastern.

o   Project Presentations (10%)

o   Each team will have 20 minutes to present what they learned when they tried to reproduce their chosen paper.

o   The slides for the presentations are due on Friday December 15, 2023 at 12:00 PM Eastern.

 

Schedule/Syllabus (Subject to Change)

Date

Lecturer

Readings

Fri Sep 8

Tina Eliassi-Rad

Overview

Tue Sep 12

Tina Eliassi-Rad

Various Representations

·      The Why, How, and When of Representations for Complex Systems

·      Non-backtracking Cycles: Length Spectrum Theory and Graph Mining Applications

Fri Sep 15

Tina Eliassi-Rad

Role Discovery

·      It's Who You Know: Graph Mining Using Recursive Structural Features

·      RolX: Structural Role Extraction & Mining in Large Graphs

·      Guided learning for Role Discovery (GLRD): Framework, Algorithms, and Applications

Tue Sep 19

Ayan Chatterjee

Graph Machine Learning for Drug Discovery + Graph Machine Learning Benchmarks

·      AI-Bind: Improving Binding Predictions for Novel Protein Targets and Ligands

·      Disentangling Node Attributes from Graph Topology for Improved Generalizability in Link Prediction

·      OGB: Open Graph Benchmark

·      DGL: Deep Graph Library

·      TGB: Temporal Graph Benchmark

·      GraphWorld

Fri Sep 22

Brennan Klein

Network Comparison and Graph Distances

·      Network Comparison and the Within-ensemble Graph Distance

·      netrd: A library for Network Reconstruction and Graph Distances

Tue Sep 26

David Liu

Node Embedding

·      STABLE: Identifying and Mitigating Instability in Embeddings of the Degenerate Core

·      Laplacian Eigenmaps for Dimensionality Reduction and Data Representation

·      node2vec: Scalable Feature Learning for Networks

·      Structural Deep Network Embedding

·      Machine Learning on Graphs: A Model and Comprehensive Taxonomy

Fri Sep 29

Team 1

Alyssa Smith & Mel Allen & Remy LeWinter

More on Representation Learning

·      Deep Graph Infomax

·      Graph Representation Learning via Graphical Mutual Information Maximization

·      [optional] Network Representation Learning: From Preprocessing, Feature Extraction to Node Embedding

Tue Oct 3

Team 2

Joey Ehlert & Mortiz Laber & Sam Dies

Low-rank Representations of Complex Networks

·      The Impossibility of Low-Rank Representations for Triangle-Rich Complex Networks

·      Node Embeddings and Exact Low-Rank Representations of Complex Networks

Fri Oct 6

Team 3

Julian Gullett & Sagar Kumar & Yixuan Liu

Collective Classification

·      Collective Classification in Network Data

·      Graph Belief Propagation Networks

·      [optional] Cautious Collective Classification

Tue Oct 10

Team 1

Alyssa Smith & Mel Allen & Remy LeWinter

Graph Neural Networks

·      Graph Neural Networks: A Review of Methods and Applications

·      Transformers are Graph Neural Networks

·      [optional] Everything is Connected: Graph Neural Networks

Fri Oct 13

Team 2

Joey Ehlert & Mortiz Laber & Sam Dies

This and That (I)

·      A Generalization of Transformer Networks to Graphs

·      Hyperbolic Graph Convolutional Neural Networks

Tue Oct 17

Tina Eliassi-Rad

Label Propagation on Graphs

·      Combining Label Propagation and Simple Models Out-performs Graph Neural Networks

·      Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification

·      [optional] Message passing all the way up

Fri Oct 20

Team 3

Julian Gullett & Sagar Kumar & Yixuan Liu

ML on Heterogeneous Graphs

·      Modeling Relational Data with Graph Convolutional Networks

·      Heterogeneous Graph Transformer

Tue Oct 24

Team 1

Alyssa Smith & Mel Allen & Remy LeWinter

W-L Graph Kernels and Power of GNNs

·      Weisfeiler-Lehman Graph Kernels

·      How Powerful are Graph Neural Networks?

·      [optional] A Reduction of a Graph to a Canonical Form and an Algebra arising during this Reduction

·      [optional] Theory of Graph Neural Networks: Representation and Learning

Fri Oct 27

Team 2

Joey Ehlert & Mortiz Laber & Sam Dies

Stability and Counting in GNNs

·      Tree Mover’s Distance: Bridging Graph Metrics and Stability of Graph Neural Networks

·      Can Graph Neural Networks Count Substructures?

Tue Oct 31

Team 3

Julian Gullett & Sagar Kumar & Yixuan Liu

GNNs for Recommendation Systems

·      Neural Graph Collaborative Filtering

·      Graph Convolutional Neural Networks for Web-Scale Recommender Systems

Fri Nov 3

Team 1

Alyssa Smith & Mel Allen & Remy LeWinter

Ayan Chatterjee will emcee this lecture.

Hypergraphs

·      Random Walks on Hypergraphs with Edge-Dependent Vertex Weights

·      Hypergraph Neural Networks

Tue Nov 7

Team 2

Joey Ehlert & Mortiz Laber & Sam Dies

David Liu will emcee this lecture.

Explainability in GNNs

·      Explainability in Graph Neural Networks: A Taxonomic Survey

·      GNNExplainer: Generating Explanations for Graph Neural Networks

Fri Nov 10

Team 3

Julian Gullett & Sagar Kumar & Yixuan Liu

Explainability & Trustworthiness of GNNs

·      GraphFramEx: Towards Systematic Evaluation of Explainability Methods for Graph Neural Networks

·      Trustworthy Graph Neural Networks: Aspects, Methods and Trends

Tue Nov 14

Team 1

Alyssa Smith & Mel Allen & Remy LeWinter

This and That (I)

·      Principal Neighbourhood Aggregation for Graph Nets

·      How does over-squashing affect the power of GNNs?

Fri Nov 17

Team 2

Joey Ehlert & Mortiz Laber & Sam Dies

Oversmooting

·      A Survey on Oversmoothing in Graph Neural Networks

·      Neural Sheaf Diffusion: A Topological Perspective on Heterophily and Oversmoothing in GNNs

Tue Nov 21

Team 3

Julian Gullett & Sagar Kumar & Yixuan Liu

Fairness & Equality

·      FAIRGEN: Towards Fair Graph Generation

·      Information Access Equality on Generative Models of Complex Networks

Fri Nov 24

No class – Thanksgiving break

Tue Nov 28

Team 1

Alyssa Smith & Mel Allen & Remy LeWinter

Learning on Signed Networks

·      Learning Signed Network Embedding via Graph Attention

·      Signed Graph Attention Networks

Fri Dec 1

Team 2

Joey Ehlert & Mortiz Laber & Sam Dies

Invariance and Equivariance

·      E(n) Equivariant Graph Neural Networks

·      Invariant and Equivariant Graph Networks

Tue Dec 5

Team 3

Julian Gullett & Sagar Kumar & Yixuan Liu

*-aware GNNs

·      Position-aware Graph Neural Networks

·      Identity-aware Graph Neural Networks

Fri Dec 8

Tina Eliassi-Rad

This lecture will be on Zoom.

This and That (II)

·      Pitfalls of Graph Neural Network Evaluation

·      Classic Graph Structural Features Outperform Factorization-Based Graph Embedding Methods on Community Labeling

Tue Dec 12

No class – NetSI Qualifying Exam Week

Fri Dec 15

Project Presentations

 

 

Notes, Policies, and Guidelines

o   You are expected to have read the assigned material before each lecture.

o   We will use Northeastern’s Canvas for announcements, assignments, and your contributions.

o   When emailing me, begin the subject line with [fa23 nets].

o   For your class project, you can use whatever programming language that you like.

o   Refresh your knowledge of the university's academic integrity policy and plagiarism. There is zero-tolerance for cheating!