Statistical Learning
A Unified Approach to Domain Incremental Learning with Memory: Theory and Algorithm
Unlike the conventional machine learning paradigms where learning is performed on a static dataset, domain incremental learning, i.e., continual learning with evolving domains, hopes to accommodate the model to the dynamically changing data distributions, while retaining the knowledge learned from previous domains [
6fee03d84375a159ecd3769ebbacae83-Supplemental-Conference.pdf
Convergence of stochastic gradient descent for non-smooth problems is a known result. For completeness, wereproduce and adapt ausual proof toour setting. Let us denote byF the class of functions fromX toY we are going to work with. Assumption 1 states that we have a well-specified modelF to estimate the median,i.e. Let us begin by controlling the estimation error.
ActiveLabeling: StreamingStochasticGradients
The workhorse of machine learning is stochastic gradient descent. To access stochastic gradients, it is common to consider iteratively input/output pairs of a training dataset. Interestingly, it appears that one does not need full supervision to access stochastic gradients, which is the main motivation of this paper. After formalizing the"activelabeling" problem, whichfocuses onactivelearningwith partial supervision, we provide a streaming technique that provably minimizes the ratio of generalization error over the number of samples.