**RandomRotate** is a type of image data augmentation where we randomly rotate the image by a degree.

A **1 x 1 Convolution** is a [convolution](https://paperswithcode.com/method/convolution) with some special properties in that it can be used for dimensionality reduction, efficient low dimensional embeddings, and applying non-linearity after convolutions. It maps an input pixel with all its channels to an output pixel which can be squeezed to a desired output depth. It can be viewed as an [MLP](https://paperswithcode.com/method/feedforward-network) looking at a particular pixel location.

Image Credit: [http://deeplearning.ai](http://deeplearning.ai)

1x1 Convolution

Network In Network

RandomRotate

**AMSGrad** is a stochastic optimization method that seeks to fix a convergence issue with [Adam](https://paperswithcode.com/method/adam) based optimizers. AMSGrad uses the maximum of past squared gradients 
$v\_{t}$ rather than the exponential average to update the parameters:

$$m\_{t} = \beta\_{1}m\_{t-1} + \left(1-\beta\_{1}\right)g\_{t} $$

$$v\_{t} = \beta\_{2}v\_{t-1} + \left(1-\beta\_{2}\right)g\_{t}^{2}$$

$$ \hat{v}\_{t} = \max\left(\hat{v}\_{t-1}, v\_{t}\right) $$

$$\theta\_{t+1} = \theta\_{t} - \frac{\eta}{\sqrt{\hat{v}_{t}} + \epsilon}m\_{t}$$

Year	2000
Data Source	CC BY-SA - https://paperswithcode.com