Neural network layers

These are the basic building blocks for neural networks. In order to use them, include the header like in the example below:

#include <metalchat/nn.h>

using namespace metalchat::nn;

Attention

template<typename T, contiguous_container Container>
class attention : public metalchat::nn::basic_layer

Embedding

template<typename T, contiguous_container Container = hardware_memory_container<T>>
class embedding : public metalchat::nn::basic_embedding<T, hardware_memory_container<T>>

Rotary positional embedding

template<typename T>
class rope : public metalchat::nn::basic_layer

This class implements Rotary Positional Embeddings (RoPE).

In this implementation we cache the frequencies for each position. When user requests an embedding with start position that is not presented in the cache, the module will recompute the cached frequencies for a range [start_pos, start_pos + max_seq_len).

Linear

template<typename T, contiguous_container Container = hardware_memory_container<T>>
class linear : public metalchat::nn::basic_linear<T, hardware_memory_container<T>>

Applies an affine linear transformation to the input data.

This module does not support bias adjustment to the input tensor, and only multiplies it (input) by the specified weight tensor. Meaning it effectively works as matrix multiplication operation.

Root mean square normalization

template<typename T, contiguous_container Container = hardware_memory_container<T>>
class rmsnorm : public metalchat::nn::basic_layer

Applies Root Mean Square Layer Normalization over a mini-batch of inputs.

Transformer

template<typename T, contiguous_container Container = hardware_memory_container<T>>
class transformer : public metalchat::nn::basic_layer