![](bg8.png)
in continuous-time and could be used to better model
physical systems without the errors incurred by sam-
pling.
ACKNOWLEDGEMENTS
We thank NVIDIA Corporation for the Quadro GPU
granted to our research group. We would also like to
thank the Pytorch developers and authors who made
their source code available to foster reproducibil-
ity in research, Henrique Lemos and Rafael Audib-
ert for their helpful discussions and help review-
ing the paper. This study was financed in part by
CNPq (Brazilian Research Council) and Coordenac¸
˜
ao
de Aperfeic¸oamento de Pessoal de N
´
ıvel Superior
(CAPES), Finance Code 001.
REFERENCES
Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez-
Gonzalez, A., Zambaldi, V. F., Malinowski, M., Tac-
chetti, A., Raposo, D., Santoro, A., Faulkner, R.,
G
¨
ulc¸ehre, C¸ ., Song, H. F., Ballard, A. J., Gilmer, J.,
Dahl, G. E., Vaswani, A., Allen, K. R., Nash, C.,
Langston, V., Dyer, C., Heess, N., Wierstra, D., Kohli,
P., Botvinick, M., Vinyals, O., Li, Y., and Pascanu,
R. (2018). Relational inductive biases, deep learning,
and graph networks. CoRR, abs/1806.01261.
Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D. J., and
Kavukcuoglu, K. (2016). Interaction networks for
learning about objects, relations and physics. In NIPS,
pages 4502–4510.
Bresson, X. and Laurent, T. (2017). Residual gated graph
convnets. CoRR, abs/1711.07553.
Chen, T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud,
D. (2018). Neural ordinary differential equations. In
NeurIPS, pages 6572–6583.
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O.,
and Dahl, G. E. (2017). Neural message passing for
quantum chemistry. In ICML, volume 70 of Pro-
ceedings of Machine Learning Research, pages 1263–
1272. PMLR.
Glorot, X. and Bengio, Y. (2010). Understanding the dif-
ficulty of training deep feedforward neural networks.
In AISTATS, volume 9 of JMLR Proceedings, pages
249–256. JMLR.org.
Gori, M., Monfardini, G., and Scarselli, F. (2005). A new
model for learning in graph domains. In IJCNN, vol-
ume 2, pages 729–734. IEEE.
Grathwohl, W., Chen, R. T. Q., Bettencourt, J., Sutskever, I.,
and Duvenaud, D. (2018). FFJORD: free-form contin-
uous dynamics for scalable reversible generative mod-
els. CoRR, abs/1810.01367.
Greff, K., Srivastava, R. K., and Schmidhuber, J. (2017).
Highway and residual networks learn unrolled itera-
tive estimation. In ICLR (Poster). OpenReview.net.
Haber, E. and Ruthotto, L. (2017). Stable architectures for
deep neural networks. CoRR, abs/1705.03341.
Hamilton, W. L., Ying, Z., and Leskovec, J. (2017). Induc-
tive representation learning on large graphs. In NIPS,
pages 1024–1034.
He, K., Zhang, X., Ren, S., and Sun, J. (2016). Deep resid-
ual learning for image recognition. In CVPR, pages
770–778. IEEE Computer Society.
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever,
I., and Salakhutdinov, R. (2012). Improving neural
networks by preventing co-adaptation of feature de-
tectors. CoRR, abs/1207.0580.
Huang, B. and Carley, K. M. (2019). Residual or
gate? towards deeper graph neural networks for
inductive graph representation learning. CoRR,
abs/1904.08035.
Kim, J., El-Khamy, M., and Lee, J. (2017). Residual
LSTM: design of a deep recurrent architecture for dis-
tant speech recognition. In INTERSPEECH, pages
1591–1595. ISCA.
Kipf, T. N. and Welling, M. (2017). Semi-supervised clas-
sification with graph convolutional networks. In ICLR
(Poster). OpenReview.net.
Li, Q., Han, Z., and Wu, X. (2018). Deeper insights into
graph convolutional networks for semi-supervised
learning. In AAAI, pages 3538–3545. AAAI Press.
Li, Y., Tarlow, D., Brockschmidt, M., and Zemel, R. S.
(2016). Gated graph sequence neural networks. In
ICLR (Poster).
Lu, Y., Zhong, A., Li, Q., and Dong, B. (2018). Beyond
finite layer neural networks: Bridging deep architec-
tures and numerical differential equations. In ICML,
volume 80 of Proceedings of Machine Learning Re-
search, pages 3282–3291. PMLR.
NT, H. and Maehara, T. (2019). Revisiting graph neural
networks: All we have is low-pass filters. CoRR,
abs/1905.09550.
Palm, R. B., Paquet, U., and Winther, O. (2018). Recurrent
relational networks. In NeurIPS, pages 3372–3382.
Poli, M., Massaroli, S., Park, J., Yamashita, A., Asama, H.,
and Park, J. (2019). Graph neural ordinary differential
equations. CoRR, abs/1911.07532.
Ruthotto, L. and Haber, E. (2018). Deep neural networks
motivated by partial differential equations. CoRR,
abs/1804.04272.
Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M.,
and Monfardini, G. (2009). The graph neural net-
work model. IEEE Transactions on Neural Networks,
20(1):61–80.
Selsam, D., Lamm, M., B
¨
unz, B., Liang, P., de Moura, L.,
and Dill, D. L. (2018). Learning a SAT solver from
single-bit supervision. CoRR, abs/1802.03685.
Veit, A., Wilber, M. J., and Belongie, S. J. (2016). Residual
networks behave like ensembles of relatively shallow
networks. In NIPS, pages 550–558.
Velickovic, P., Cucurull, G., Casanova, A., Romero, A., Li
`
o,
P., and Bengio, Y. (2018). Graph attention networks.
In ICLR (Poster). OpenReview.net.
Wang, Y. and Tian, F. (2016). Recurrent residual learning
for sequence classification. In EMNLP, pages 938–
943. The Association for Computational Linguistics.
Wu, Y. and He, K. (2018). Group normalization. In ECCV
(13), volume 11217 of Lecture Notes in Computer Sci-
ence, pages 3–19. Springer.
ICAART 2021 - 13th International Conference on Agents and Artificial Intelligence
126