Abstract:
There is an increasing interest in developing new models for graph classification problem that serves as a common benchmark for evaluation and comparison of GNNs and graph kernels. To ensure a fair comparison of the models several commonly used datasets exist and current assessments and conclusions rely on the validity of these datasets. However, as we show in this paper majority of these datasets contain isomorphic copies of the data points, which can lead to misleading conclusions. For example, the relative ranking of the graph models can change substantially if we remove isomorphic graphs in the test set. To mitigate this we present several results. We show that explicitly incorporating the knowledge of isomorphism in the datasets can significantly boost the performance of any graph model. Finally, we re-evaluate commonly used graph models on refined graph datasets and provide recommendations for designing new datasets and metrics for graph classification problem.
