[D] Weighing softmax predictions based on the validation set confusion matrix, does it make sense? • r/MachineLearning

Suppose I have a classification neuralnet for which I compute the confusion matrix on the validation set after my network has converged. What ways are there of using this matrix to reliably increase the accuracy on unseen data? I know of setting a per-class minimum confidence threshold. But would it make sense to reponder the softmax predictions knowing that some class A is often misclassified as B by the network etc...?