**Reduction-B** is an image model block used in the [Inception-v4](https://paperswithcode.com/method/inception-v4) architecture.

**Models Genesis**, or **Generic Autodidactic Models**, is a self-supervised approach for learning 3D image representations. The objective of Models Genesis is to learn a common image representation that is transferable and generalizable across diseases, organs, and modalities.  It consists of an encoder-decoder architecture with skip connections in between, and is trained to learn a common image representation by restoring the original sub-volume $x\_{i}$ (as ground truth) from the transformed one $\bar{x}\_{i}$ (as input), in which the reconstruction loss (MSE) is computed between the model prediction $x'\_{0}$ and ground truth $x\_{i}$. Once trained, the encoder alone can be fine-tuned for target classification tasks; while the encoder and decoder together can be fine-tuned for target segmentation tasks.

Models Genesis

Reduction-B

Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning

**Holographic Reduced Representations** are a simple mechanism to represent an associative array of key-value pairs in a fixed-size vector. Each individual key-value pair is the same size as the entire associative array; the array is represented by the sum of the pairs. Concretely, consider a complex vector key $r = (a\_{r}[1]e^{iφ\_{r}[1]}, a\_{r}[2]e^{iφ\_{r}[2]}, \dots)$, which is the same size as the complex vector value x. The pair is "bound" together by element-wise complex multiplication, which multiplies the moduli and adds the phases of the elements:

$$ y = r \otimes x $$

$$ y =  \left(a\_{r}[1]a\_{x}[1]e^{i(φ\_{r}[1]+φ\_{x}[1])}, a\_{r}[2]a\_{x}[2]e^{i(φ\_{r}[2]+φ\_{x}[2])}, \dots\right) $$

Given keys $r\_{1}$, $r\_{2}$, $r\_{3}$ and input vectors $x\_{1}$, $x\_{2}$, $x\_{3}$, the associative array is:

$$c = r\_{1} \otimes x\_{1} + r\_{2} \otimes x\_{2} + r\_{3} \otimes x\_{3} $$

where we call $c$ a memory trace. Define the key inverse:

$$ r^{-1} = \left(a\_{r}[1]^{−1}e^{−iφ\_{r}[1]}, a\_{r}[2]^{−1}e^{−iφ\_{r}[2]}, \dots\right) $$

To retrieve the item associated with key $r\_{k}$, we multiply the memory trace element-wise by the vector $r^{-1}\_{k}$. For example: 

$$ r\_{2}^{−1} \otimes c = r\_{2}^{-1} \otimes \left(r\_{1} \otimes x\_{1} + r\_{2} \otimes x\_{2} + r\_{3} \otimes x\_{3}\right) $$

$$ r\_{2}^{−1} \otimes c = x\_{2} + r^{-1}\_{2} \otimes \left(r\_{1} \otimes x\_{1} + r\_{3} \otimes x3\right) $$

$$ r\_{2}^{−1} \otimes c = x\_{2} + noise $$

The product is exactly $x\_{2}$ together with a noise term. If the phases of the elements of the key vector are randomly distributed, the noise term has zero mean.

Source: [Associative LSTMs](https://arxiv.org/pdf/1602.03032.pdf)

Source	Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Year	2000
Data Source	CC BY-SA - https://paperswithcode.com