Media
[D] Are Model Sizes Approaching Human Cortical Numbers?
As someone with a neuroscience background, it's interesting to see the parallels between the parameter size & number of neurons in the recent GPT-3 unveiling versus the cortical regions of the brain responsible for language processing. Estimates put cortical neurons at 25B, and there is somewhere on the order of 7K connections between each neuron (further pruned over time), putting the total number of cortical parameters at somewhere close to 175T. Let's say language processing areas are 1/5th of this, the rest being dedicated to vision, higher order processing, executive function, etc. This puts the language generating portion of the human brain at maybe 35T parameters - yet we're capable of producing such fantastic results with GPT-3's 175B, a number which is 1/200th of the size. Of course, the two are not directly equatable - GPT-3, for example, has'read' several million books' worth of content at this point, whereas the average human has read just a few dozen by the time they're able and language-producing, but it's still interesting to reason about.
Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling
Tan, Hao Hao, Herremans, Dorien
High-level musical qualities (such as emotion) are often abstract, subjective, and hard to quantify. Given these difficulties, it is not easy to learn good feature representations with supervised learning techniques, either because of the insufficiency of labels, or the subjectiveness (and hence large variance) in human-annotated labels. In this paper, we present a framework that can learn high-level feature representations with a limited amount of data, by first modelling their corresponding quantifiable low-level attributes. We refer to our proposed framework as Music FaderNets, which is inspired by the fact that low-level attributes can be continuously manipulated by separate "sliding faders" through feature disentanglement and latent regularization techniques. High-level features are then inferred from the low-level representations through semi-supervised clustering using Gaussian Mixture Variational Autoencoders (GM-VAEs). Using arousal as an example of a high-level feature, we show that the "faders" of our model are disentangled and change linearly w.r.t. the modelled low-level attributes of the generated output music. Furthermore, we demonstrate that the model successfully learns the intrinsic relationship between arousal and its corresponding low-level attributes (rhythm and note density), with only 1% of the training set being labelled. Finally, using the learnt high-level feature representations, we explore the application of our framework in style transfer tasks across different arousal states. The effectiveness of this approach is verified through a subjective listening test.
Choiceworx Receives VC Funding Round, Industry Veteran Frank Casale Appointed Chief Revenue Officer – Cognitive Business News
CIOs, CTOs, and heads of digital transformation are struggling to deliver high quality 24/7 automated help desk and service desk support and trying to address the monitoring, management, and repair of their RPA bots. These challenges validate our business model and the market need for the ChoiceWORX AI platform offerings,
Restoring Common Sense In An Age Of Experts & Artificial Intelligence
Information, data, and choice continue to expand in exponential ways leading to a constant and never-ending sense of drowning. If there was too much to know yesterday, there's more today, and tomorrow will have yet more. There is no way to catch up. And yet the promise of optimized decision making in the face of this overwhelming choice is as alluring as ever. All indicators suggest the availability of a perfect selection, one that does more than merely providing a good outcome.