Active Mini-Batch Sampling using Repulsive Point Processes

  • Cheng Zhang | Microsoft Research

The performance of stochastic gradient descent (SGD) can be improved by actively selecting mini-batches. In this work, we explore active mini-batch selection using repulsive point processes. This simultaneously introduces active bias and leads to stochastic gradients with lower variance. We show empirically that our approach improves over standard SGD both in terms of convergence speed as well as final model performance.

Regardez suivant