Summary: Deep generative models have been praised for their ability to learn smooth latent representations of images, text, and audio, which can then be used to generate new, plausible data. Motivated by these success stories, there has been a surge of interest in developing deep generative models for automated molecule design. However, these models face several difficulties due to the unique characteristics of molecular graphs – their underlying structure is not Euclidean or grid-like, they remain isomorphic under permutation of the nodes’ labels, and they come with a different number of nodes and edges. In this paper, we first propose a novel variational autoencoder for molecular graphs, whose encoder and decoder are specially designed to account for the above properties by means of several technical innovations. Moreover, in contrast with the state of the art, our decoder is able to provide the spatial coordinates of the atoms of the molecules it generates. Then, we develop a gradient-based algorithm to optimize the decoder of our model so that it learns to generate molecules that maximize the value of certain property of interest and, given any arbitrary molecule, it is able to optimize the spatial configuration of its atoms for greater stability. Experiments reveal that our variational autoencoder can discover plausible, diverse and novel molecules more effectively than several state of the art models. Moreover, for several properties of interest, our optimized decoder is able to identify molecules with property values 121% higher than those identified by several state of the art methods based on Bayesian optimization and reinforcement learning.

##### MSC:

68T05 | Learning and adaptive systems in artificial intelligence |

##### Keywords:

drug design; molecule discovery; deep generative models; variational autoencoders; geometric deep learning
PDF
BibTeX
XML
Cite

\textit{B. Samanta} et al., J. Mach. Learn. Res. 21, Paper No. 114, 33 p. (2020; Zbl 07255145)

Full Text:
Link

##### References:

[1] | Albert-L´aszl´o Barab´asi and R´eka Albert. Emergence of scaling in random networks.science, 286(5439): 509-512, 1999. |

[2] | Hanjun Dai, Yingtao Tian, Bo Dai, Steven Skiena, and Le Song. Syntax-directed variational autoencoder for structured data. InICLR, 2018. |

[3] | Nicola De Cao and Thomas Kipf. Molgan: An implicit generative model for small molecular graphs.arXiv preprint arXiv:1805.11973, 2018. |

[4] | MJ Frisch, GW Trucks, HB Schlegel, GE Scuseria, MA Robb, JR Cheeseman, G Scalmani, V Barone, B Mennucci, GA Petersson, et al. Gaussian 09, revision a. 02; gaussian, inc: Wallingford, ct, 2009.There is no corresponding record for this reference, 2015. |

[5] | Niklas Gebauer, Michael Gastegger, and Kristof Sch¨utt. Symmetry-adapted generation of 3d point sets for the targeted discovery of molecules. InNeurIPS, 2019. |

[6] | Rafael G´omez-Bombarelli, David Duvenaud, Jos´e Miguel Hern´andez-Lobato, Jorge Aguilera-Iparraguirre, Timothy D Hirzel, Ryan P Adams, and Al´an Aspuru-Guzik. Automatic chemical design using a datadriven continuous representation of molecules.arXiv preprint arXiv:1610.02415, 2016. |

[7] | Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling, Carlos Outeiral, Pedro Luis Cunha Farias, and Al´an Aspuru-Guzik. Objective-reinforced generative adversarial networks (organ) for sequence generation models.arXiv preprint arXiv:1705.10843, 2017. |

[8] | William Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs.NIPS, 2017. |

[9] | John J Irwin, Teague Sterling, Michael M Mysinger, Erin S Bolstad, and Ryan G Coleman. Zinc: a free tool to discover chemistry for biology.Journal of chemical information and modeling, 52(7):1757-1768, 2012. |

[10] | Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Junction tree variational autoencoder for molecular graph generation.arXiv preprint arXiv:1802.04364, 2018. |

[11] | Donald R Jones, Matthias Schonlau, and William J Welch. Efficient global optimization of expensive blackbox functions.Journal of Global optimization, 13(4):455-492, 1998. |

[12] | Jack Kiefer, Jacob Wolfowitz, et al. Stochastic estimation of the maximum of a regression function.The Annals of Mathematical Statistics, 23(3):462-466, 1952. |

[13] | Diederik P Kingma and Max Welling. Auto-encoding variational bayes.arXiv preprint arXiv:1312.6114, 2013. |

[14] | Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In Proceedings of the 4th International Conference on Learning Representations, 2016a. |

[15] | Thomas N Kipf and Max Welling. Variational graph auto-encoders.arXiv preprint arXiv:1611.07308, 2016b. |

[16] | Solomon Kullback and Richard A Leibler. On information and sufficiency.The annals of mathematical statistics, 22(1):79-86, 1951. |

[17] | Matt J Kusner, Paige, Brooks, and Jos´e Miguel Hern´andez-Lobato. Grammar variational autoencoder.arXiv preprint arXiv:1703.01925, 2017. |

[18] | Tao Lei, Wengong Jin, Regina Barzilay, and Tommi Jaakkola. Deriving neural architectures from sequence and graph kernels.ICML, 2017. |

[19] | Jure Leskovec, Deepayan Chakrabarti, Jon Kleinberg, Christos Faloutsos, and Zoubin Ghahramani. Kronecker graphs: An approach to modeling networks.Journal of Machine Learning Research, 11(Feb): 985-1042, 2010. |

[20] | Qi Liu, Miltiadis Allamanis, Marc Brockschmidt, and Alexander L Gaunt. Constrained graph variational autoencoders for molecule design.arXiv preprint arXiv:1805.09076, 2018. |

[21] | Kenneth M Merz, Dagmar Ringe, and Charles H Reynolds.Drug design: structure-and ligand-based approaches. Cambridge University Press, 2010. |

[22] | Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. InNIPS, 2013. |

[23] | Steven M Paul, Daniel S Mytelka, Christopher T Dunwiddie, Charles C Persinger, Bernard H Munos, Stacy R Lindborg, and Aaron L Schacht. How to improve r&d productivity: the pharmaceutical industry’s grand challenge.Nature reviews Drug discovery, 9(3):203, 2010. |

[24] | Trang Pham, Truyen Tran, Dinh Q Phung, and Svetha Venkatesh. Column networks for collective classification. InProceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pages 2485-2491, 2017. |

[25] | Pavel G Polishchuk, Timur I Madzhidov, and Alexandre Varnek. Estimation of the size of drug-like chemical space based on gdb-17 data.Journal of computer-aided molecular design, 27(8):675-679, 2013. |

[26] | Raghunathan Ramakrishnan, Pavlo O Dral, Matthias Rupp, and O Anatole Von Lilienfeld. Quantum chemistry structures and properties of 134 kilo molecules.Scientific data, 1:140022, 2014. |

[27] | Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. Stochastic backpropagation and approximate inference in deep generative models.arXiv preprint arXiv:1401.4082, 2014. |

[28] | Lars Ruddigkeit, Ruud Van Deursen, Lorenz C Blum, and Jean-Louis Reymond. Enumeration of 166 billion organic small molecules in the chemical universe database gdb-17.Journal of chemical information and modeling, 52(11):2864-2875, 2012. |

[29] | Bidisha Samanta, Abir De, Gourhari Jana, Pratim Kumar Chattaraj, , Niloy Ganguly, and Manuel GomezRodriguez. Nevae: A deep generative model for molecular graphs.AAAI, 2019. |

[30] | Marwin HS Segler, Mike Preuss, and Mark P Waller. Planning chemical syntheses with deep neural networks and symbolic ai.Nature, 555(7698):604, 2018. |

[31] | Martin Simonovsky and Nikos Komodakis. Graphvae: Towards generation of small graphs using variational autoencoders.arXiv preprint arXiv:1802.03480, 2018. |

[32] | Edward Snelson and Zoubin Ghahramani. Sparse gaussian processes using pseudo-inputs. InNeurIPS, 2006. 22 |

[33] | Ronald J Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229-256, 1992. |

[34] | Jiaxuan You, Bowen Liu, Zhitao Ying, Vijay Pande, and Jure Leskovec. Graph convolutional policy network for goal-directed molecular graph generation. InNeurIPS, 2018a. |

This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. It attempts to reflect the references listed in the original paper as accurately as possible without claiming the completeness or perfect precision of the matching.