Tensor library#

This is the fundamental library of MetalChat. It provides a convenient interface for management of multi-dimensional matrices. This library could be used like following:

#include <metalchat/tensor.h>

Tensor#

template<typename T, std::size_t N, contiguous_container Container = random_memory_container<T>> class tensor : public metalchat::basic_tensor #

A multi-dimensional matrix containing elements of a single data type.

A tensor can be constructed from the std::initializer_list or by copying data from the various sources (see various tensor constructs below);

auto T = tensor({{1.0f, -1.0f}, {1.0f, -1.0f}});
std::cout << T << std::endl;
// out:
// [[1.0 -1.0],
//  [1.0 -1.0]], sizes=(2, 2)

A tensor of specific data type can be constructed by specified a concrete tensor type with a template parameter T and/or allocator , memory_container, or hardware_accelerator, or tensor creation operation:

auto T = zeros<int32_t>({2, 4});
std::cout << T << std::endl;
// out:
// [[0, 0, 0, 0],
//  [0, 0, 0, 0]], sizes=(2, 4)

auto I = full<float>({2, 4}, 1.0f, hardware_accelerator());
std::cout << I << std::endl;
// out:
// [[1.0, 1.0, 1.0, 1.0],
//  [1.0, 1.0, 1.0, 1.0]], sizes=(2, 4)

For more information about building tensors, see Advanced creation documentation.

Warning

Depending on the selected tensor constructor and allocator, tensor could copy data.

Template Parameters:

T – data type of the tensor elements (float, int, etc.).
N – dimensionality of the tensor.
Container – a particular implementation of the storage for the tensor elements.

Public Types

using value_type = T #: Alias of the tensor type.

using pointer_type = T*#: Pointer to the tensor type.

using container_type = Container #: Container type storing the data of the tensor.

using container_pointer = std::shared_ptr<container_type>#: Pointer to the container type storing the data of the tensor.

using iterator = tensor_iterator<T, N>#: Contiguous iterator of the tensor data.

using const_iterator = const iterator #: Contiguous constant iterator of the tensor data.

using accessor_type = tensor_accessor#: A tensor accessor type.

Public Functions

inline tensor()#

Constructs an empty tensor (zero size and empty data).

This constructor does not allocate a container, therefore direct access to data using tensor::data_ptr is invalid.

tensor(tensor &&other) noexcept = default#: The move constructor. Constructs a tensor with the contents of other.

tensor(const tensor &other) noexcept = default#: The copy constructor. Constructs a tensor with the contents of other.

template<allocator_t<T> Allocator = scalar_memory_allocator<T>> inline tensor(const T &value, Allocator alloc = Allocator())#

Constructs a new tensor with a scalar value. By default, method is using a scalar_memory_allocator, but an arbitrary typed allocator could be used.

auto T = tensor<float, 0, scalar_memory_container<float>>(3.0f);
// Same as:
auto S = scalar<float>(3.0f);

Parameters:

value – the value to initialize a tensor with.
alloc – allocator to use for all memory allocations of this container.

inline tensor(std::initializer_list<T> data)#

Constructs a new 1-dimensional tensor and initializes it with the given values.

auto T = tensor({1.0f, 2.0f, 3.0f, 4.0f});

Parameters:: data – initial data of the tensor.

inline tensor(std::initializer_list<std::initializer_list<T>> data)#

Constructs a new 2-dimensional tensor and initializes it with the given values.

Method creates a tensor with dimensions that fill all specified values, missing values are filled with T().

auto T = tensor({{1.0f, 2.0f, 3.0f}, {3.0f, 4.0f}});
std::cout << T << std::endl;
// out:
// [[1.0, 2.0, 3.0],
//  [3.0, 4.0, 0.0]], sizes=(2, 3)

Parameters:: data – initial data of the tensor.

template<std::forward_iterator ForwardIt, allocator_t<T> Allocator> inline tensor(ForwardIt first, ForwardIt last, Allocator alloc)#

Constructs a new empty tensor with the sizes specified by iterators [first, last).

The distance std::distance(first, last) between iterators should be equal to the tensor dimensionality N.

auto sizes = std::vector({4, 3, 6, 7});
auto T = tensor<float, 4>(sizes.begin(), sizes.end(), random_memory_allocator<float>());

Parameters:

first, last – the pair of iterators defining the range of dimensions of the tensor to copy from.
alloc – allocator to use for all memory allocations of this container.

template<std::forward_iterator ForwardIt, allocator_t<T> Allocator = random_memory_allocator<T>> inline tensor(ForwardIt first, ForwardIt last, value_type *data, Allocator alloc = Allocator())#

Constructs a new tensor with the size defined by the contents of the range first, last.

The distance std::distance(first, last) between iterators should be equal to the tensor dimensionality N.

The tensor container is initialized with the contents pointed by data, therefore the underlying storage should be a contiguously allocated memory. Depending on the specified allocator, data could be copied, or used transparently as is.

auto sizes = std::vector<std::size_t>({10, 2, 5});
auto contents = std::vector<float>(100, 4.0f);

auto T = tensor(sizes.begin(), sizes.end(), contents.data());

Parameters:

first, last – the pair of iterators defining the range of dimensions of the tensor to copy from.
data – the contents that will be used as data for the tensor.
alloc – allocator to use for all memory allocations of this container.

template<std::forward_iterator ForwardIt> inline tensor(ForwardIt first, ForwardIt last, const container_pointer &data)#

Constructs a new tensor with the sizes specified by iterators [first, last), and with contents of the specified container data.

The distance std::distance(first, last) between iterators should be equal to the tensor dimensionality N.

auto sizes = std::vector<std::size_t>({4, 5});

auto alloc = random_memory_allocator<float>();
auto container_ptr = alloc.allocate(20);

auto T = tensor<float>(sizes.begin(), sizes.end(), container_ptr);

Parameters:

first, last – the pair of iterators defining the range of dimensions of the tensor to copy from.
data – initial data of the tensor.

template<allocator_t<T> Allocator = random_memory_allocator<T>> inline tensor(const std::span<std::size_t, N> sizes, Allocator alloc = Allocator())#

Constructs an empty tensor of the specified size with uninitialized container allocated with the given allocator.

auto sizes = std::vector<std::size_t>({4, 5});
auto alloc = std::random_memory_allocator<float>();

auto T = tensor<float>(std::span<std::size_t, 2>(sizes.begin(), 2), alloc);

Parameters:

sizes – a sequence of unsigned integers defining the shape of the tensor.
alloc – allocator to use for all memory allocations of this container.

inline tensor(const std::span<std::size_t, N> sizes, const container_pointer &data)#

Constructs a new tensor with the specified sizes with contents of the specified container data. The constructor does not validate that all elements of the tensor are within the specified container.

auto sizes = std::vector<std::size_t>({4, 5});

auto alloc = random_memory_allocator<float>();
auto container_ptr = alloc.allocate(20);

auto T = tensor<float>(std::span<std::size_t, 2>(sizes.begin(), 2), container_ptr);

Parameters:

sizes – a sequence of unsigned integers defining the shape of the tensor.
data – initial data of the tensor.

template<allocator_t<T> Allocator = random_memory_allocator<T>> inline tensor(std::size_t (&&sizes)[N], Allocator alloc = Allocator())#

Constructs an empty tensor with the uninitialized contents of the specified size.

auto alloc = random_memory_allocator<float>();
auto T = tensor<float>({3, 4, 5}, alloc);

Parameters:

sizes – a sequence of unsigned integers defining the shape of the tensor.
alloc – allocator to use for all memory allocations of this container.

inline tensor(std::size_t (&&sizes)[N], const container_pointer &data)#

Constructs a new tensor with contents of the specified container data. The constructor does not validate that all elements of the tensor are within the container.

auto alloc = random_memory_allocator<float>();
auto container_ptr = alloc.allocate(120);

auto T = tensor<float>({1, 2, 3, 4, 5}, container_ptr);

Parameters:

sizes – a sequence of unsigned integers defining the shape of the tensor.
data – initial data of the tensor.

inline tensor(const accessor_type &access, const container_pointer &data)#

Constructs a new tensor with the layout specified by accessor and contents specified in a container data. The constructor does not validate that all elements of the tensors are within the container.

auto alloc = random_memory_allocator<float>();
auto container_ptr = alloc.allocate(120);

auto accessor = tensor_accessor({3, 4, 10});
auto T = tensor<float>(accessor, container_ptr);

Parameters:

access – a tensor accessor defining the layout of data in the container.
data – initial data of the tensor.

inline virtual const std::type_info &dtype() const#: Returns type information about tensor elements data type.

inline virtual std::size_t dimensions() const#: Returns the number of dimension of the tensor.

inline virtual const void *data() const#: Returns a constant byte-aligned pointer to the first element of the tensor.

inline virtual void *data()#: Returns a byte-aligned pointer to the underlying data.

inline pointer_type data_ptr() noexcept#: Returns a pointer to the first element of the tensor.

inline const pointer_type data_ptr() const noexcept#: Returns a pointer to the first element of the tensor.

inline virtual std::size_t stride(std::size_t dim) const#

Returns the stride of the specified tensor dimension.

auto T = empty<float>({2, 5});
std::cout << T.stride(0) << std::endl;
// out: 5
std::cout << T.stride(1) << std::endl;
// out: 1

Tensor layout#

template<uint32_t N> struct tensor_layout#

The layout of the tensor. This type is used for encoding tensor as a parameter of the Metal kernel. Tensor layout is passed as a first argument after the pointer to the data (Metal buffer).

The same structure is available in the Metal library and is used to unpack the memory into corresponding parts of the layout schema (sizes, strides, offsets).

Public Members

uint32_t sizes[N]#: Sizes of a tensor.

uint32_t strides[N]#: Strides of a tensor data.

uint32_t offsets[N]#: Offsets of a tensor data.

Tensor iterator#

template<typename T, std::size_t N> class tensor_iterator#

An iterator of multi-dimensional matrix returning elements in a row-major (lexicographic) order.

Usually tensor_iterator is constructed using methods tensor::begin and tensor::end. But could also be used as a standalone entity.

auto T = rand<float>({3, 4});
for (auto it = T.begin(); it != T.end(); ++it) {
    std::cout << (*it) << std::endl;
}

Template Parameters:

T – data type of the tensor elements (float, int, etc.)
N – dimensionality of the underlying data container.

Public Types

using value_type = T #: Alias of the tensor type.

using iterator_category = std::forward_iterator_tag#: Declares that tensor_iterator is a forward iterator.

using iterator = tensor_iterator<T, N>#: Iterator type.

using reference = value_type&#: Reference type of the tensor element.

using pointer = value_type*#: Pointer type of the tensor element.

using difference_type = std::ptrdiff_t#: Specifies a distance type between two iterators.

using container_pointer = std::shared_ptr<basic_container>#: Pointer to the container type storing the data of the tensor.

Public Functions

template<immutable_tensor Tensor> inline tensor_iterator(const Tensor &tensor, difference_type start)#

Creates a new tensor_iterator instance.

Initializes iterator with a pointer to the tensor’s container. Therefore tensor iterator is still valid even after destruction of the original tensor.

template<immutable_tensor Tensor> inline tensor_iterator(const Tensor &tensor)#: Creates a new tensor_iterator that points to the first element of the tensor.

tensor_iterator(const iterator &it) = default#: The default constructor of tensor_iterator.

inline iterator &operator++()#: Advances the iterator forward.

inline iterator &operator+(difference_type n)#: Advances the iterator forward by n elements.

inline reference operator*()#

Returns a reference to the current tensor element.

The end-of-data iterator dereference always returns the first element of the tensor.

inline pointer operator->()#

Returns a pointer to the current tensor element.

The end-of-data iterator dereference always returns the first element of the tensor.

inline bool operator==(const iterator &rhs) const#

Compares two tensor_iterators.

Method compares data pointers of the underlying tensor containers and current position of the iterator. Note that iterators are considered equal even when they are traversing tensor data with a different view (different strides and offsets).

inline bool operator!=(const iterator &rhs) const#

Compares two tensor_iterators.

The implementation is negation of the tensor_iterator::operator==.

Basic tensor#

class basic_tensor#

The abstract class interface to an unbounded set of classes encapsulating tensors.

The primary usage of the classes derived from basic_tensor are using them in nn::basic_layer parameters. For example, you can construct a multi-layer model and access it’s parameters through a unified interface.

Subclassed by metalchat::tensor< T, N, Container >, metalchat::tensor< T, N, Container >

Public Functions

virtual const std::type_info &dtype() const = 0#: Returns type information about tensor elements data type.

virtual const void *data() const = 0#: Returns a constant byte-aligned pointer to the underlying data.

virtual void *data() = 0#: Returns a byte-aligned pointer to the underlying data.

virtual std::shared_ptr<basic_container> container_ptr() const = 0#: Returns a pointer to the underlying data container of the tensor.

virtual std::size_t dimensions() const = 0#: Returns the number of dimension of the tensor.

virtual std::size_t size(std::size_t dim) const = 0#

Returns the size of the specified tensor dimension.

Parameters:: dim – the dimension for which to retrieve the size.

virtual void set_size(std::size_t dim, std::size_t size) = 0#

Sets the size at the specified tensor dimension.

Parameters:

dim – the target dimension.
size – the size to set.

virtual const std::span<std::size_t> sizes() const = 0#: Returns the sizes of the tensor.

virtual std::size_t stride(std::size_t dim) const = 0#

Returns the stride at the specified tensor dimension.

Parameters:: dim – the dimension for which to retrieve the stride.

virtual void set_stride(std::size_t dim, std::size_t stride) = 0#

Sets the stride at the specified tensor dimension.

Parameters:

dim – the target dimension.
stride – the stride to set.

virtual const std::span<std::size_t> strides() const = 0#: Returns strides of the tensor.

virtual std::size_t offset(std::size_t dim) const = 0#

Returns the container offset of the specified tensor dimension.

Parameters:: dim – the dimension for which to retrieve the offset.

virtual void set_offset(std::size_t dim, std::size_t offset) = 0#

Sets the offset at the specified tensor dimension.

Parameters:

dim – the target dimension.
offset – the offset to set.

virtual const std::span<std::size_t> offsets() const = 0#: Returns the offsets of the tensor container.

virtual std::size_t numel() const = 0#: Returns the total number of elements in the tensor.

inline virtual ~basic_tensor()#: The tensor destructor.

Shared tensor#

template<immutable_tensor Tensor> class shared_tensor_ptr#

Public Functions

inline std::size_t numel() const#: See tensor::numel.

inline pointer_type data_ptr()#: See tensor::data_ptr.

inline const pointer_type data_ptr() const#: See tensor::data_ptr.

inline std::size_t size(std::size_t dim) const#: See tensor::size.

inline const std::span<std::size_t> sizes() const#: See tensor::sizes.

inline const std::span<std::size_t, N> shape() const#: See tensor::shape.

inline std::size_t stride(std::size_t dim) const#: See tensor::stride.

inline const std::span<std::size_t> strides() const#: See tensor::strides.

inline std::size_t offset(std::size_t dim) const#: See tensor::offset.

inline const std::span<std::size_t> offsets() const#: See tensor::offsets.

inline iterator begin()#: See tensor::begin.

inline iterator end()#: See tensor::end.

inline const_iterator begin() const#: See tensor::begin.

inline const_iterator end() const#: See tensor::end.

template<convertible_to_slice... SliceTypes> inline auto index_select(const SliceTypes&... slices)#: See tensor::index_select.

inline auto expand_dims(std::size_t dim) const#: See tensor::expand_dims.

template<std::size_t M> inline auto view(int (&&dims)[M]) const#: See tensor::view.

template<std::size_t M> inline auto view(const std::span<int, M> dims) const#: See tensor::view.

template<std::size_t M> inline auto view(const std::span<std::size_t, M> dims) const#: See tensor::view.

template<std::size_t M> inline auto flatten() const#: See tensor::flatten.

inline shared_tensor_ptr narrow(std::size_t dim, std::size_t start, std::size_t length) const#: See tensor::transpose.

inline shared_tensor_ptr transpose(const std::size_t (&&dims)[N]) const#: See tensor::transpose.

inline tensor_layout<N> layout() const#: See tensor::layout.

Public Static Functions

static inline constexpr std::size_t dim()#: See tensor::dim.