A spiking neural network inspired by neuroscience and psychology for Western mode- and key-conditioned music learning and composition

Dataemia
11 Min Read


  • Todd, P. M. A connectionist approach to algorithmic composition. Comput. Music. J. 13, 27–43 (1989).


    Google Scholar
     

  • Mozer, M. C. Neural network music composition by prediction: Exploring the benefits of psychoacoustic constraints and multi-scale processing. Connect. Sci. 6, 247–280. https://doi.org/10.1080/09540099408915726 (1994).


    Google Scholar
     

  • Eck, D. & Schmidhuber, J. A first look at music composition using LSTM recurrent neural networks. Istituto Dalle Molle Di Studi Sull Intelligenza Artificiale 103, 48 (2002).


    Google Scholar
     

  • Waite, E. Generating long-term structure in songs and stories (2016). https://magenta.tensorflow.org/2016/07/15/lookback-rnn-attention-rnn/.

  • Boulanger-Lewandowski, N., Bengio, Y. & Vincent, P. Modeling temporal dependencies in high-dimensional sequences: Application to polyphonic music generation and transcription. arXiv preprint arXiv:1206.6392 (2012).

  • Hadjeres, G., Pachet, F. & Nielsen, F. Deepbach: a steerable model for bach chorales generation. In International Conference on Machine Learning, 1362–1371 (PMLR, 2017).

  • Briot, J.-P., Hadjeres, G. & Pachet, F.-D. Deep Learning Techniques for Music Generation, vol. 1 (Springer, 2020).

  • Ji, S., Yang, X. & Luo, J. A survey on deep learning for symbolic music generation: Representations, algorithms, evaluations, and challenges. ACM Comput. Surv. 56, 1–39 (2023).


    Google Scholar
     

  • Wang, Z. et al. Structured representation learning for polyphonic music. In International Society for Music Information Retrieval Conference, 368–375 (2020).

  • Shiqi Wei, G. X. Learning long-term music representations via hierarchical contextual constraints. In International Society for Music Information Retrieval Conference, 738–745 (2022).

  • Roberts, A., Engel, J., Raffel, C., Hawthorne, C. & Eck, D. A hierarchical latent vector model for learning long-term structure in music. In International Conference on Machine Learning, 4364–4373 (PMLR, 2018).

  • Trieu, N. & Keller, R. M. Jazzgan: Improvising with generative adversarial networks. In In Proc. of the 6th International Workshop on Musical Metacreation (MUME) (2018).

  • Dong, H.-W., Hsiao, W.-Y., Yang, L.-C. & Yang, Y.-H. Musegan: Multi-track sequential generative adversarial networks for symbolic music generation and accompaniment. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018).

  • Yang, L.-C., Chou, S.-Y. & Yang, Y.-H. Midinet: A convolutional generative adversarial network for symbolic-domain music generation. In International Society for Music Information Retrieval Conference, 324–331 (2017).

  • Huang, Y.-S. & Yang, Y.-H. Pop music transformer: Beat-based modeling and generation of expressive pop piano compositions. In Proceedings of the 28th ACM International Conference on Multimedia, 1180–1188 (2020).

  • Hernandez-Olivan, C., Hernandez-Olivan, J. & Beltrán, J. R. A survey on artificial intelligence for music generation: Agents, domains and perspectives. arXiv.org https://doi.org/10.48550/arxiv.2210.13944 (2022).

  • Payne, C. Musenet (2019). http://openai.com/blog/musenet.

  • Xu, W., McAuley, J., Berg-Kirkpatrick, T., Dubnov, S. & Dong, H.-W. Generating symbolic music from natural language prompts using an llm-enhanced dataset. arXiv preprint arXiv:2410.02084 (2024).

  • Wang, Y. et al. Notagen: Advancing musicality in symbolic music generation with large language model training paradigms (2025). arXiv:2502.18008.

  • Samek, W., Wiegand, T. & Müller, K.-R. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models (2017). arXiv:1708.08296.

  • Lake, B. M., Ullman, T. D., Tenenbaum, J. B. & Gershman, S. J. Building machines that learn and think like people. Behav. Brain Sci. 40, e253. https://doi.org/10.1017/S0140525X16001837 (2017).


    Google Scholar
     

  • Zador, A. et al. Catalyzing next-generation artificial intelligence through neuroai. Nat. Commun. 14, 1597. https://doi.org/10.1038/s41467-023-37180-x (2023).


    Google Scholar
     

  • Krumhansl, C. L. Cognitive Foundations of Musical Pitch (Oxford University Press, 1990).

  • Temperley, D. What’s key for key? The Krumhansl–Schmuckler key-finding algorithm reconsidered. Music. Percept. 17, 65–100 (1999).


    Google Scholar
     

  • Gilboa, A. & Marlatte, H. Neurobiology of schemas and schema-mediated memory. Trends Cogn. Sci. 21, 618–631 (2017).


    Google Scholar
     

  • van Kesteren, M. T., Ruiter, D. J., Fernández, G. & Henson, R. N. How schema and novelty augment memory formation. Trends Neurosci. 35, 211–219. https://doi.org/10.1016/j.tins.2012.02.001 (2012).


    Google Scholar
     

  • Preston, A. R. & Eichenbaum, H. Interplay of hippocampus and prefrontal cortex in memory. Curr. Biol. 23, R764–R773 (2013).


    Google Scholar
     

  • Van Kesteren, M. T., Ruiter, D. J., Fernández, G. & Henson, R. N. How schema and novelty augment memory formation. Trends Neurosci. 35, 211–219 (2012).


    Google Scholar
     

  • Brod, G., Werkle-Bergner, M. & Shing, Y. L. The influence of prior knowledge on memory: a developmental cognitive neuroscience perspective. Front. Behav. Neurosci. 7, 139 (2013).


    Google Scholar
     

  • Tubridy, S. & Davachi, L. Medial temporal lobe contributions to episodic sequence encoding. Cereb. Cortex 21, 272–280. https://doi.org/10.1093/cercor/bhq092 (2011).


    Google Scholar
     

  • McAndrews, M. P. & Milner, B. The frontal cortex and memory for temporal order. Neuropsychologia 29, 849–859. https://doi.org/10.1016/0028-3932(91)90051-9 (1991).


    Google Scholar
     

  • Meier, B. et al. Implicit task sequence learning in patients with Parkinson’s disease, frontal lesions and amnesia: The critical role of fronto–striatal loops. Neuropsychologia 51, 3014–3024. https://doi.org/10.1016/j.neuropsychologia.2013.10.009 (2013).


    Google Scholar
     

  • Jenkins, L. J. & Ranganath, C. Prefrontal and medial temporal lobe activity at encoding predicts temporal context memory. J. Neurosci. 30, 15558–15565. https://doi.org/10.1523/JNEUROSCI.1337-10.2010 (2010).


    Google Scholar
     

  • Vuust, P., Heggli, O. A., Friston, K. J. & Kringelbach, M. L. Music in the brain. Nat. Rev. Neurosci. 23, 287–305 (2022).


    Google Scholar
     

  • Purves, D. et al. The auditory cortex. In Neuroscience, 2nd edn (Sinauer Associates, 2001).

  • Sankaran, N., Leonard, M. K., Theunissen, F. & Chang, E. F. Encoding of melody in the human auditory cortex. Sci. Adv. 10, eadk0010 (2024).

  • McDermott, J. H. & Oxenham, A. J. Music perception, pitch, and the auditory system. Curr. Opin. Neurobiol. 18, 452–463 (2008).


    Google Scholar
     

  • Merchant, H., Harrington, D. L. & Meck, W. H. Neural basis of the perception and estimation of time. Annu. Rev. Neurosci. 36, 313–336 (2013).


    Google Scholar
     

  • Gupta, D. S. Processing of sub-and supra-second intervals in the primate brain results from the calibration of neuronal oscillators via sensory, motor, and feedback processes. Front. Psychol. 5, 816 (2014).


    Google Scholar
     

  • MacDonald, C. J., Lepage, K. Q., Eden, U. T. & Eichenbaum, H. Hippocampal, “time cells’’ bridge the gap in memory for discontiguous events. Neuron 71, 737–749 (2011).


    Google Scholar
     

  • Liang, Q., Zeng, Y. & Xu, B. Temporal-sequential learning with a brain-inspired spiking neural network and its application to musical memory. Front. Comput. Neurosci. 14, 51. https://doi.org/10.3389/fncom.2020.00051 (2020).


    Google Scholar
     

  • Liang, Q. & Zeng, Y. Stylistic composition of melodies based on a brain-inspired spiking neural network. Front. Syst. Neurosci. 15, 639484. https://doi.org/10.3389/fnsys.2021.639484 (2021).


    Google Scholar
     

  • Zeng, Y. et al. Braincog: A spiking neural network based, brain-inspired cognitive intelligence engine for brain-inspired AI and brain simulation. Patterns 4, 100789. https://doi.org/10.1016/j.patter.2023.100789 (2023).


    Google Scholar
     

  • Izhikevich, E. M. Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569. https://doi.org/10.1109/TNN.2003.820440 (2003).


    Google Scholar
     

  • Spitzer, N. C. Electrical activity in early neuronal development. Nature 444, 707–712 (2006).


    Google Scholar
     

  • Hua, J. Y. & Smith, S. J. Neural activity and the dynamics of central nervous system development. Nat. Neurosci. 7, 327–332. https://doi.org/10.1038/nn1215 (2004).


    Google Scholar
     

  • Zheng, J. Q. & ming Poo, M. Calcium signaling in neuronal motility. Annu. Rev. Cell Dev. Biol. 23, 375–404. https://doi.org/10.1146/annurev.cellbio.23.090506.123221 (2007).

  • Bi, G.-Q. & Poo, M.-M. Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464–10472. https://doi.org/10.1523/JNEUROSCI.18-24-10464.1998 (1998).


    Google Scholar
     

  • Hannon, E. E. & Trainor, L. J. Music acquisition: effects of enculturation and formal training on development. Trends Cogn. Sci. 11, 466–472 (2007).


    Google Scholar
     

  • Herholz, S. C. & Zatorre, R. J. Musical training as a framework for brain plasticity: Behavior, function, and structure. Neuron 76, 486–502 (2012).


    Google Scholar
     

  • Dubovsky, I., Yevseev, S. V., Sposobin, I. V. & Sokolov, V. V. Textbook of Harmony (Parts I & II) (People’s Music Publishing House, 1991).

  • Huang, H. Exercises and Answers for Harmony Textbook, vol. I (People’s Music Publishing House, 2011).

  • Cuthbert, M. S. & Ariza, C. music21: A toolkit for computer-aided musicology and symbolic music data. In Proceedings of the 11th International Society for Music Information Retrieval Conference (ISMIR 2010) (2010).

  • Yang, L.-C. & Lerch, A. On the evaluation of generative models in music. Neural Comput. Appl. 32, 4773–4784 (2020).


    Google Scholar
     

  • Dong, H.-W., Chen, K., McAuley, J. & Berg-Kirkpatrick, T. Muspy: A toolkit for symbolic music generation. arXiv preprint arXiv:2008.01951 (2020).

  • Hernandez-Olivan, C. & Beltran, J. R. Musicaiz: A python library for symbolic music generation, analysis and visualization. SoftwareX 22, 101365 (2023).


    Google Scholar
     

  • Temperley, D. Music and Probability (MIT Press, 2007).

  • Temperley, D. A probabilistic model of melody perception. Cogn. Sci. 32, 418–444. https://doi.org/10.1080/03640210701863332 (2008).


    Google Scholar
     

  • Krumhansl, C. et al. Cross-cultural music cognition: cognitive methodology applied to north sami yoiks. Cognition 76, 13–58 (2000).


    Google Scholar
     

  • You, S. et al. Physics of life reviews. 53, 131–133. https://doi.org/10.1016/j.plrev.2025.03.002 (2025).

  • Koelsch, S., Vuust, P. & Friston, K. Predictive processes and the peculiar case of music. Trends Cogn. Sci. 23, 63–77. https://doi.org/10.1038/s41583-022-00578-5 (2019).


    Google Scholar
     



  • Source link

    Share This Article
    Leave a Comment

    Leave a Reply

    Your email address will not be published. Required fields are marked *

    error: Content is protected !!