Asynchronous operations#

Asynchronously invocable#

template<typename T> concept asynchronously_invocable#

#include <future.h>

The asynchronously_invocable concept specifies that a type T can be called asynchronously, and result could be either awaited through future or through callback (or both).

Effectively, this type is used as a opaque wrapper around a task that computes result of a future tensor. The implementation of operator() should not block the thread and exit fast. The implementation of make_ready_at_thread_exit should block the thread, until the result is not computed.

Future tensor#

template<typename T, std::size_t N> class future_tensor#

A tensor associated with a computation task, which result is not ready yet.

A future tensor holds a pointer to the pre-allocated on-device memory, and a task that is responsible for filling that memory. Future tensor is immutable, meaning that it’s content cannot be modified before the completion of an associated task.

Since the tensor is immutable, any immutable operation (which does not modify the underlying data) could be executed. Such operations include: transposition, slicing, narrowing, dimensionality expansion, etc.

Public Types

using value_type = result_type::value_type#: Alias of the tensor type.

using pointer_type = result_type::pointer_type#: Pointer to the tensor type.

using container_type = result_type::container_type#: Container type storing the data of the tensor.

using container_pointer = result_type::container_pointer#: Pointer to the container type storing the data of the tensor.

using iterator = result_type::iterator#: Contiguous iterator of the tensor data.

using const_iterator = result_type::const_iterator#: Contiguous constant iterator of the tensor data.

Public Functions

template<typename U, std::size_t M> inline future_tensor(result_type result, future_tensor<U, M> future)#: Create a future tensor that expects completion of other future tensor.

template<typename U, std::size_t M> inline future_tensor(future_tensor result, future_tensor<U, M> future)#

Create a future tensor that expects completion of two other future tensors

A future tensor result becomes the result of the new tensor completion. This operation in non-destructible, so both result and future tensors could be awaited separately from this tensor.

template<asynchronously_invocable Task> inline future_tensor(future_tensor result, Task &&task)#

Create a future tensor that expects completion of the specified task.

A new tensor will wait for completion of both self task and a new asynchronously invocable task, and only then makes result accessible.

#include <future>

struct noop_async_func {
    std::promise<void> promise;

    noop_async_func()
    : promise()
    {
        promise.set_value();
    }

    std::shared_future<void>
    operator()()
    {
        return std::shared_future(promise.get_future());
    }

    // In this example, callback is executed immediately since the promise
    // is ready on the class instantiation.
    std::shared_future<void>
    operator()(std::function<void()> callback)
    {
        callback();
        return std::shared_future(promise.get_future());
    }

    void
    make_ready_at_thread_exit()
    { }
};


auto accelerator = hardware_accelerator(16);
auto T = future_tensor(empty<float>({3, 2}, accelerator));

// Tensor `F` will be waiting for the completion of the specified
// asynchronously_invocable function.
auto F = future_tensor(T, noop_async_func());

inline future_tensor(result_type result)#: A naive future tensor that does not wait for any task and returns result immediately.

inline future_tensor()#: A default future tensor constructor.

inline result_type get()#: Waits for (by calling future_tensor::wait) until the shared tensor is ready, then retrieves the value stored in the shared state.

inline result_type get_nowait() const#: Return the result tensor without waiting of the associated operation completion.

Warning

Since the operation is not awaited, the data container of the returned tensor could (and will) be populated asynchronously to the main application thread.

inline void wait()#: Blocks until the result becomes available.

inline std::size_t numel() const#: See tensor::numel.

inline const tensor_accessor &accessor() const#: See tensor_accessor.

inline container_type &container() const#: See tensor::container.

inline std::shared_ptr<basic_container> container_ptr() const#: See tensor::container_ptr.

inline pointer_type data_ptr()#: See tensor::data_ptr.

Warning

Tensor must be awaited before accessing data with an iterator.

inline const pointer_type data_ptr() const#: See tensor::data_ptr.

Warning

Tensor must be awaited before accessing data with an iterator.

inline std::size_t size(std::size_t dim) const#: See tensor::size.

inline const std::span<std::size_t> sizes() const#: See tensor::sizes.

inline const std::span<std::size_t, N> shape() const#: See tensor::shape.

inline std::size_t stride(std::size_t dim) const#: See tensor::stride.

inline const std::span<std::size_t> strides() const#: See tensor::strides.

inline std::size_t offset(std::size_t dim) const#: See tensor::offset.

inline const std::span<std::size_t> offsets() const#: See tensor::offsets.

inline iterator begin()#: See tensor::begin.

Warning

Tensor must be awaited before accessing data with an iterator.

inline iterator end()#: See tensor::end.

inline const_iterator begin() const#: See tensor::begin.

Warning

Tensor must be awaited before accessing data with an iterator.

inline const_iterator end() const#: See tensor::end.

inline future_tensor<T, N + 1> expand_dims(std::size_t dim) const#: See tensor::expand_dims.

template<std::size_t M> inline future_tensor<T, M> view(int (&&dims)[M]) const#: See tensor::view.

template<std::size_t M> inline future_tensor<T, M> view(const std::span<int, M> dims) const#: See tensor::view.

template<std::size_t M> inline future_tensor<T, M> view(const std::span<std::size_t, M> dims) const#: See tensor::view.

template<std::size_t M> inline future_tensor<T, M> flatten() const#: See tensor::flatten.

inline future_tensor narrow(std::size_t dim, std::size_t start, std::size_t length) const#: See tensor::narrow.

inline future_tensor transpose(const std::size_t (&&dims)[N]) const#: See tensor::transpose.

inline tensor_layout<N> layout() const#: See tensor::layout.

Public Static Functions

static inline constexpr std::size_t dim()#: See tensor::dim.