This paper addresses the challenge of handling extreme class imbalance, defined here as a situation in which negative examples, conventionally the majority, outnumber positive examples, usually the ones of most interest, by a factor of 500 or more (in other words, have an imbalance ratio (IR) 500). Such problems are not in fact uncommon, and arise in application areas such as fraud detection [1] and cheminformatics [2]. We make use of two methods, that tackle different, but complementary, aspects of the class imbalance problem: ASTra, a novel, adaptive, asymmetric output layer activation function, which makes the correct classification of minority examples easier. A loss function based on an approximated confusion matrix, which aggressively targets the misclassification of minority examples. Our proposed methods have the advantage of being easy to implement and integrate into the workflow of any model that makes binary predictions normally generated by a sigmoid activation (transfer) function. In addition, the paper presents a new means of monitoring training and validation performance, especially valuable in cases of high class imbalance, that could potentially be used with any training regime, independently of the proposed methods.