In particular, modelling equations for different musical structural parameters were defined, namely: rhythmic structure, sound level/perceptual loudness, and tempo, modality, pitch register and pitch contour. Through the creation of two agents, these parameters are incorporated, preserving the original equations in this new mode of generation. As the integration of affective models forms the basis of this exploration, how these human affective states are modelled follows the valence-arousal model introduced by Russell (1980). Formerly, music psychology literature labelled affective states using a categorical model, suggesting these stem from a finite number of monopolar universal basic affects. However, currently various two or three-dimensional models have been more universally adopted, with Russell's circumplex model of affect being commonly used, due to its ability to represent the complexities of affect. This approach employs valence (pleasure vs. displeasure) and arousal (high vs. low energy) as its dimensions, and is used in the research. The reinforcement learning problem in the context of generating musical code based on valence-arousal coordinates involves training an agent to select sequences of code that correspond to desired affective qualities.