Differences between LDA, QDA and Gaussian Naive Bayes classifiers

While digging in the details of classical classification methods, I found sparse information about the similarities and differences of Gaussian Naive Bayes (GNB), Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA). This post centralises the information I found for the next learner. Summary: All three methods are a specific instance of The Bayes Classifier, they all deal with continuous Gaussian predictors, they differ in the assumptions they makes about the relationships amongst predictors, and across classes (i.e. the way they specify the covariance matrices). We have a set X of p predictors, and a discrete response variable Y (the class) taking values k {1, …, K}, for a sample of n observations. We encounter a new observation for which we know the values of the predictors X, but not the class Y, so we would like to make a guess about Y based on the information we have (our sample).