[mlpack-git] master: Add update formula. (da10c65)

[gitdub at big.cc.gt.atl.ga.us]

Repository : https://github.com/mlpack/mlpack

On branch  : master
Link       : https://github.com/mlpack/mlpack/compare/6e98f6d5e61ac0ca861f0a7c3ec966076eccc50e...7de290f191972dd41856b647249e2d24d2bf029d

>---------------------------------------------------------------

commit da10c65470105e18f4b67efea1aefb49fbe38f2d
Author: Marcus Edel <marcus.edel at fu-berlin.de>
Date:   Sat Jun 20 17:42:54 2015 +0200

    Add update formula.


>---------------------------------------------------------------

da10c65470105e18f4b67efea1aefb49fbe38f2d
 src/mlpack/methods/ann/optimizer/rmsprop.hpp | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/src/mlpack/methods/ann/optimizer/rmsprop.hpp b/src/mlpack/methods/ann/optimizer/rmsprop.hpp
index 1a25b60..c40aec3 100644
--- a/src/mlpack/methods/ann/optimizer/rmsprop.hpp
+++ b/src/mlpack/methods/ann/optimizer/rmsprop.hpp
@@ -15,7 +15,14 @@ namespace ann /** Artificial Neural Network. */ {
 
 /**
  * RmsProp is an optimizer that utilizes the magnitude of recent gradients to
- * normalize the gradients.
+ * normalize the gradients. In its basic form, given a step rate \f$ \gamma \f$
+ * and a decay term \f$ \alpha \f$ we perform the following updates:
+ *
+ * \f[
+ * r_t &=& (1 - \gamma) f'(\Delta_t)^2 + \gammar_{t - 1} \\
+ * v_{t + 1} &=& \frac{\alpha}{\sqrt{r_t}}f'(\Delta_t) \\
+ * \Delta_{t + 1} &=& \Delta_t - v_{t + 1}
+ * \f]
  *
  * For more information, see the following.
  *