Layer library#

Basic layer#

class basic_layer#

Layer is a basic building block of neural networks in MetalChat. A layer specifies a set of (trainable) parameters it uses for computation and a set of upstream layers, used within a layer computation logic.

Subclassed by metalchat::nn::basic_embedding< T, hardware_memory_container< T > >, metalchat::nn::basic_linear< T, hardware_memory_container< T > >, metalchat::nn::attention< T, Container, Cache >, metalchat::nn::basic_embedding< T, Container >, metalchat::nn::basic_linear< T, Container >, metalchat::nn::feed_forward< T, Container, Activation >, metalchat::nn::layer_array< Layer >, metalchat::nn::llama3< T, Container >, metalchat::nn::rmsnorm< T, Container >, metalchat::nn::rope< T >, metalchat::nn::sink_cache< T >, metalchat::nn::transformer< T, Container, Activation >, metalchat::quantization::lora_adaptor< T, Container >

Public Functions

basic_layer(const hardware_accelerator &accelerator)#

Construct a layer that is a associated with the specified hardware accelerator.

The layer is constructed with a default - . (dot) delimiter.

char delimiter() const#: Return a delimiter used to join names of layers and parameters of nested layers.

const hardware_accelerator &accelerator() const#: Get a constant reference to the hardware accelerator.

hardware_accelerator &accelerator()#: Get a reference to the hardware accelerator.

layer_type &layer(const std::string &name)#: Return a reference to the registered layer by the specified name.

const layer_type &layer(const std::string &name) const#: Return a const reference to the registered layer by the specified name.

parameter_type &parameter(const std::string &name)#

Return a reference to the registered parameter by the specified name.

This method also supports recursive lookup of the parameter within children layers if the name contains a basic_layer::delimiter() const.

const parameter_type &parameter(const std::string &name) const#

Returns a const reference to the registered parameter.

See parameter(const std::string&)

std::vector<named_parameter> parameters(bool recurse = true) const#

Return a set of parameters with fully-qualified names. Parameters of different layers are separated using a configured basic_layer::delimiter() const symbol.

If you want to return only parameters of the current layer and drop upstream parameters, you could call this method with recurse = false.

parameter_pointer &parameter_ptr(const std::string &name)#

Return a pointer to the registered parameter by the specified name.

This method also supports recursive lookup of the parameter within children layers if the name contains a delimiter.

const parameter_pointer &parameter_ptr(const std::string &name) const#

Returns a conster pointer to the registered parameter.

See parameter_ptr(const std::string&).

template<immutable_tensor Tensor> inline void set_parameter(const std::string &name, Tensor &&tensor)#

Set value to the registered layer parameter.

When the specified parameter is not found, the method throws an exception. The method supports assignment of the nested parameters.

Example:

using namespace metalchat;

auto accelerator = hardware_accelerator(32);
auto linear = nn::linear<float>(accelerator);

linear.set_parameter("weight", empty<float>({4, 4}, accelerator));

template<immutable_tensor Tensor> inline shared_tensor_ptr<Tensor> register_parameter(const std::string &name, Tensor &&tensor)#

Add a parameter to the layer.

The parameter can be accessed using basic_layer::parameter method and updated with basic_layer::set_parameter method respectively.

A common practice is registering parameters of the layers that could be updated externally (loaded from a file, or stored after inference):

using namespace metalchat;

struct custom_layer : public layer {
    // Declare parameters here.
    shared_tensor<float, 3> weight;

    custom_layer(hardware_accelerator accelerator)
    : layer(accelerator)
    {
        weight = register_parameter("weight", empty<float>({10, 4, 3}, accelerator));
    }
};

template<immutable_tensor Tensor> inline shared_tensor_ptr<Tensor> register_parameter(const std::string &name, const shared_tensor_ptr<Tensor> &tensor_ptr)#

Add a parameter to the layer.

This method shared ownership of the tensor (parameter) with the caller. Consider the following example, where the parameter is constructed with the basic layer using delegated constructors, and then registered in the body of the constructor:

using namespace metalchat;

struct custom_layer : public layer {
    // Declare parameters here.
    shared_tensor<float, 3> weight;

    custom_layer(hardware_accelerator accelerator)
    : layer(accelerator),
      weight()
    {
        weight = register_parameter("weight", full<float>({5, 4, 2}, 4.0, accelerator));
    }
};

Template Parameters:

Tensor – a type of the parameter’s tensor.

Parameters:

name – a name of the parameter to register.
tensor_ptr – a shared pointer to the parameter value.

template<mutable_layer Layer, typename ...Args> inline indirect_layer<Layer> register_layer(const std::string &name, Args&&... args)#

Register an upstream layer for the current layer. The layer could be accessed using the given name using basic_layer::layer method.

The registry of layers owns the upstream layer, and the method returns a object pointing to that owned layer.

A common practice is registering upstream layers within a downstream layer constructor like in the example below.

using namespace metalchat;

struct custom_layer : public layer {
    // Declare upstream layers here.
    nn::indirect_layer<nn::linear<float>> linear1;
    nn::indirect_layer<nn::linear<float>> linear2;

   custom_layer(hardware_accelerator accelerator)
   : layer(accelerator)
   {
      // Register layers here.
      linear1 = register_layer<nn::linear<float>>("linear1");
      linear2 = register_layer<nn::linear<float>>("linear2");
   }
};

Note

You can explore a variety of different layers in Neural network layers .

Template Parameters:

Layer – a layer type to create.
Args – types of parameters for a Layer’s constructor.

Parameters:

name – a name of the layer to register.
args – arguments used to construct the layer instance.

template<mutable_layer Layer> inline indirect_layer<Layer> register_layer(const std::string &name, const indirect_layer<Layer> &layer)#

Register an upstream layer for the current layer.

Does the same a basic_layer::register_layer but with already built indirect layer.

template<std::invocable<named_parameter> Function> inline void apply(Function fn, bool recurse = true)#

Apply a function to every parameters of the layer.

This method traverses all parameters in breadth-first way when recurse parameter is set to true. Otherwise, only parameters of the current layer are visited.

Indirect layer#

template<mutable_layer Layer> class indirect_layer#

A Wrapper around a shared pointer for arbitrary layer implementation provides invocable functionality for Layer implementations.

Template Parameters:: Layer – A wrapped layer type.

Public Functions

inline indirect_layer()#: Construct a shared layer with no managed layer, i.e. empty shared_ptr.

inline indirect_layer(const std::shared_ptr<layer_type> &r)#: Construct a shared layer which shares ownership of the layer managed by r.

template<typename ...Args> inline auto operator()(Args&&... args)#

Invoke the stored layer target with the parameters args.

Effectively does f(std::forward<Args>(args)...);, where f is the target layer.

inline std::shared_ptr<layer_type> get() const#: Return the raw shared pointer to the layer.

inline layer_type *operator->() noexcept#: Dereference the stored pointer to the Layer.

inline layer_type &operator*() noexcept#: Dereference the stored pointer to the Layer.

inline explicit operator bool() const noexcept#: Checks if the *this stores a null layer pointer.

const hardware_accelerator &accelerator() const#: Get a constant reference to the hardware accelerator.

hardware_accelerator &accelerator()#: Get a reference to the hardware accelerator.

basic_layer &layer(const std::string &name)#: Return a reference to the registered layer by the specified name.

const basic_layer &layer(const std::string &name) const#: Return a const reference to the registered layer by the specified name.

template<immutable_tensor Tensor> void set_parameter(const std::string &name, Tensor &&tensor)#

Set value to the registered layer parameter.

When the specified parameter is not found, the method throws an exception. The method supports assignment of the nested parameters.

Example:

using namespace metalchat;

auto accelerator = hardware_accelerator(32);
auto linear = nn::linear<float>(accelerator);

linear.set_parameter("weight", empty<float>({4, 4}, accelerator));

parameter_type &parameter(const std::string &name)#

Return a reference to the registered parameter by the specified name.

This method also supports recursive lookup of the parameter within children layers if the name contains a basic_layer::delimiter() const.

std::vector<named_parameter> parameters(bool recurse = true) const#

Return a set of parameters with fully-qualified names. Parameters of different layers are separated using a configured basic_layer::delimiter() const symbol.

If you want to return only parameters of the current layer and drop upstream parameters, you could call this method with recurse = false.

Polymorphic layer#

template<mutable_layer Layer> class polymorphic_layer#

Layer array#

template<mutable_layer Layer> class layer_array : public metalchat::nn::basic_layer #

Sequential container of layers.

layer_array can be indexed like a random access container, but layers it contains are properly registered, and will be visible by all basic_layer methods.

struct my_layer : public basic_layer {
    // Step 2. Define helper types.
    using Linear = nn::linear<float>;
    using LinearArray = nn::layer_array<Linear>;

    // Step 2. Create a layer array as a type member.
    nn::indirect_layer<LinearArray> linears;

    // Step 3. Register a layer array as a sub-layer.
    my_layer(hardware_accelerator& accelerator)
    : layer(accelerator)
    {
        linears = register_layer<LinearArray>("linears");

        for (std::size_t i = 0; i < 10; i++) {
            // Step 3. Initialize layers within an array.
            linears.emplace_back(10, 10, accelerator);
        }
    }

    template<immutable_tensor2_t<float> Input>
    auto
    operator()(Input input)
    {
        for (std::size_t i = 0; i < 10; i++) {
            // Step 4. Use layers as a regular random-access array.
            input = linears[i / 2](input) + linears[i](input);
        }
        return input;
    }
};

Note

You can access elements of the layer array through basic_layer::parameter(const std::string&) const method by using the following syntax: array.0.

Template Parameters:: Layer – a type of the layers this module stores.

Public Functions

inline layer_array(const hardware_accelerator &accelerator)#: The layer array constructor.

inline reference operator[](size_type pos)#

Returns a reference to the pos-element of the layer array.

Parameters:: pos – the position of a layer in the array.

inline reference at(size_type pos)#

Returns a reference to the pos-element of the layer array.

Parameters:: pos – the position of a layer in the array.

inline reference back()#: Returns a reference to the last element in the container.

inline const_reference back() const#: Returns a const reference to the last element in the container.

inline const_reference operator[](size_type pos) const#

Returns a constant reference to the pos-element of the layer array.

Parameters:: pos – the position of a layer in the array.

inline const_reference at(size_type pos) const#

Returns a constant reference to the pos-element of the layer array.

Parameters:: pos – the position of a layer in the array.

inline void push_back(const indirect_layer<Layer> &layer)#

Appends an existing layer to the end of the container.

Parameters:: layer – the layer to append.

template<typename ...Args> inline void emplace_back(Args&&... args)#

Appends a new layer to the end of the container. The arguments args... are forwarded to the layer constructor as std::forward<Args>(args)....

Template Parameters:: Args – argument types to forward to the constructor of the layer.
Parameters:: args – arguments to forward to the constructor of the layer.

inline size_type size() const#: Returns the number of elements in the container.