Training Neural Nets: a Hacker's Perspective

Along the way, I'll share personal commentary, stories from established deep learning practitioners, and code snippets. Let's start by looking at the common points that can fail a neural network. There are three categorical features here: Sex, Has_Masters, and Has_Bachelors. You may one-hot encode to better represent the relationship or you may just keep them as they are. There are two continuous features in the dataset: Age and Bounties. They vary largely in scale, so you would want to standardize their scales. There are several ways to initialize the weights in a neural network. You can start with all zeros (which isn't advisable, and we will see it a second), you can randomly initialize them, or you can choose a technique like Xavier initialization or He initialization.