Layer library#

Basic layer#

class basic_layer : public std::enable_shared_from_this<basic_layer>#

Layer is a basic building block of neural networks in MetalChat. A layer specifies a set of (trainable) parameters it uses for computation and a set of upstream layers, used within a layer computation logic.

Subclassed by metalchat::nn::basic_embedding< T, hardware_memory_container< T > >, metalchat::nn::basic_linear< T, hardware_memory_container< T > >, metalchat::nn::attention< T, Container >, metalchat::nn::basic_embedding< T, Container >, metalchat::nn::basic_linear< T, Container >, metalchat::nn::feed_forward< T, Container >, metalchat::nn::layer_array< Layer >, metalchat::nn::llama3< T, Container, Cache >, metalchat::nn::rmsnorm< T, Container >, metalchat::nn::rope< T >, metalchat::nn::sink_cache< T >, metalchat::nn::transformer< T, Container >, metalchat::quantization::lora_adaptor< T, Container >

Public Types

using parameter_pointer = std::shared_ptr<basic_tensor>#: A shared pointer to the basic tensor.

using layer_pointer = std::shared_ptr<basic_layer>#: A shared pointer to the basic layer.

Public Functions

basic_layer(const hardware_accelerator &accelerator)#: Construct a layer that is a associated with the specified hardware accelerator.

const hardware_accelerator &accelerator() const#: Get a constant reference to the hardware accelerator.

hardware_accelerator &accelerator()#: Get a reference to the hardware accelerator.

template<typename Layer, typename ...Args> inline indirect_layer<Layer> register_layer(const std::string &name, Args&&... args)#

Register an upstream layer for the current layer. The layer could be accessed using the given name using basic_layer::get_layer method.

The registry of layers owns the upstream layer, and the method returns a object pointing to that owned layer.

A common practice is registering upstream layers within a downstream layer constructor like in the example below.

using namespace metalchat;

struct custom_layer : public basic_layer {
    // Declare upstream layers here.
    indirect_layer<nn::linear<float>> linear1;
    indirect_layer<nn::linear<float>> linear2;

   custom_layer(hardware_accelerator accelerator)
   : basic_layer(accelerator)
   {
      // Register layers here.
      linear1 = register_layer<nn::linear<float>>("linear1");
      linear2 = register_layer<nn::linear<float>>("linear2");
   }
};

Note

You can explore a variety of different layers in Neural network layers .

basic_layer &get_layer(const std::string &name) const#: Get upstream layer by name. This method does not perform recursive lookup and only returns layers registered at the current layer. If layer is not registered, method throws exception.

template<immutable_tensor Tensor> inline shared_tensor_ptr<Tensor> register_parameter(const std::string &name, Tensor &&tensor)#

Add a parameter to the layer.

The parameter can be accessed using basic_layer::get_parameter method and updated with basic_layer::set_parameter method respectively.

A common practice is registering parameters of the layers that could be updated externally (loaded from a file, or stored after inference):

using namespace metalchat;

struct custom_layer : public basic_layer {
    // Declare parameters here.
    shared_tensor<float, 3> weight;

    custom_layer(hardware_accelerator accelerator)
    : basic_layer(accelerator)
    {
        weight = register_parameter("weight", empty<float>({10, 4, 3}, accelerator));
    }
};

template<immutable_tensor Tensor> inline shared_tensor_ptr<Tensor> register_parameter(const std::string &name, const shared_tensor_ptr<Tensor> &tensor_ptr)#

Add a parameter to the layer.

This method shared ownership of the tensor (parameter) with the caller. Consider the following example, where the parameter is constructed with the basic layer using delegated constructors, and then registered in the body of the constructor:

using namespace metalchat;

struct custom_layer : public basic_layer {
    // Declare parameters here.
    shared_tensor<float, 3> weight;

    custom_layer(hardware_accelerator accelerator)
    : basic_layer(accelerator),
      weight(full<float>({5, 4, 2}, 4.0, accelerator))
    {
        register_parameter("weight", weight);
    }
};

template<immutable_tensor Tensor> inline void set_parameter(const std::string &name, Tensor &&tensor)#

Set value to the registered layer parameter.

When the specified parameter is not found, the method throws an exception. The method supports assignment of the nested parameters.

Example:

using namespace metalchat;

auto accelerator = hardware_accelerator(32);
auto linear = nn::linear<float>(accelerator);

linear.set_parameter("weight", empty<float>({4, 4}, accelerator));

parameter_pointer get_parameter(const std::string &name) const#

Return a pointer to the registered parameter by the specified name.

This method also supports recursive lookup of the parameter within children layers if the name contains a dot (‘.’) delimiter.

const parameter_container get_parameters(bool recurse = true) const#

Return a set of parameters with fully-qualified names. Parameters of different layers are separated using dot (‘.’) delimiter symbol.

If you want to return only parameters of the current layer and drop upstream parameters, you could call this method with recurse = false.

template<std::invocable<named_parameter> Function> inline void apply(Function fn, bool recurse = true) const#

Apply a function to every parameters of the layer.

This method traverses all parameters in breadth-first way when recurse parameter is set to true. Otherwise, only parameters of the current layer are visited.

Indirect layer#

template<typename Layer> class indirect_layer#

A Wrapper around a shared pointer for arbitrary layer implementation provides invocable functionality for Layer implementations.

Public Functions

inline indirect_layer()#: Construct a shared layer with no managed layer, i.e. empty shared_ptr.

inline indirect_layer(std::shared_ptr<layer_type> r)#: Construct a shared layer which shares ownership of the layer managed by r.

template<typename ...Args> inline auto operator()(Args&&... args)#

Invoke the stored layer target with the parameters args.

Effectively does f(std::forward<Args>(args)...);, where f is the target layer.

inline std::shared_ptr<layer_type> get() const#: Return the raw shared pointer to the layer.

inline layer_type *operator->() noexcept#: Dereference the stored pointer to the Layer.

inline layer_type &operator*() noexcept#: Dereference the stored pointer to the Layer.

Polymorphic layer#

template<typename Layer> class polymorphic_layer#

Layer array#

template<typename Layer> class layer_array : public metalchat::nn::basic_layer #

Sequential container of layers.

layer_array can be indexed like a random access container, but layers it contains are properly registered, and will be visible by all basic_layer methods.

struct my_layer : public basic_layer {
    // Step 1. Create a layer array as a type member.
    layer_array<nn::linear<float>> linears;

    // Step 2. Register a layer array as a sub-layer.
    my_layer(const hardware_accelerator& accelerator)
    : basic_layer(accelerator),
      linears(*register_layer("linears", layer_array<nn::linear<float>>(accelerator)))
    {
        for (std::size_t i = 0; i < 10; i++) {
            // Step 3. Initialize layers within an array.
            linears.emplace_back(10, 10, accelerator);
        }
    }

    template<immutable_tensor2_t<float> Input>
    auto
    operator()(Input input)
    {
        for (std::size_t i = 0; i < 10; i++) {
            // Step 4. Use layers as a regular random-access array.
            input = linears[i / 2](input) + linears[i](input);
        }
        return input;
    }
};

Note

You can access elements of the layer array through basic_layer::get_parameter method by using the following syntax: array.0.

Template Parameters:: Layer – a type of the layers this module stores.

Public Functions

inline layer_array(const hardware_accelerator &accelerator)#: The layer array constructor.

inline reference operator[](size_type pos)#

Returns a reference to the pos-element of the layer array.

Parameters:: pos – the position of a layer in the array.

inline reference at(size_type pos)#

Returns a reference to the pos-element of the layer array.

Parameters:: pos – the position of a layer in the array.

inline const_reference operator[](size_type pos) const#

Returns a constant reference to the pos-element of the layer array.

Parameters:: pos – the position of a layer in the array.

inline const_reference at(size_type pos) const#

Returns a constant reference to the pos-element of the layer array.

Parameters:: pos – the position of a layer in the array.

inline void push_back(const indirect_layer<Layer> &layer)#

Appends an existing layer to the end of the container.

Parameters:: layer – the layer to append.

template<typename ...Args> inline void emplace_back(Args&&... args)#

Appends a new layer to the end of the container. The arguments args... are forwarded to the layer constructor as std::forward<Args>(args)....

Template Parameters:: Args – argument types to forward to the constructor of the layer.
Parameters:: args – arguments to forward to the constructor of the layer.

inline size_type size() const#: Returns the number of elements in the container.