Modern deep learning (DL) applications are built using DL libraries and frameworks such as TensorFlow and PyTorch. These frameworks have complex parameters and tuning them to obtain good training and inference performance is challenging for typical users, such as DL developers and data scientists. Manual tuning requires deep knowledge of the user-controllable parameters of DL frameworks as well as the underlying hardware. It is a slow and tedious process, and it typically delivers sub-optimal solutions. In this paper, we treat the problem of tuning parameters of DL frameworks to improve training and inference performance as a black-box optimization problem. We then investigate applicability and effectiveness of Bayesian optimization (BO), genetic algorithm (GA), and Nelder-Mead simplex (NMS) to tune the parameters of TensorFlow's CPU backend. While prior work has already investigated the use of Nelder-Mead simplex for a similar problem, it does not provide insights into the applicability of other more popular algorithms. Towards that end, we provide a systematic comparative analysis of all three algorithms in tuning TensorFlow's CPU backend on a variety of DL models. Our findings reveal that Bayesian optimization performs the best on the majority of models. There are, however, cases where it does not deliver the best results.

Cochlear implants (CIs) are an effective intervention for individuals with severe-to-profound sensorineural hearing loss. Currently, no tuning procedure exists that can fully exploit the technology. We propose online unsupervised algorithms to learn features from the speech of a severely-to-profoundly hearing-impaired patient round-the-clock and compare the features to those learned from the normal hearing population using a set of neurophysiological metrics. Experimental results are presented. The information from comparison can be exploited to modify the signal processing in a patient’s CI to enhance his audibility of speech.