Interpreter library#

Interpreter#

class interpreter#

Each message submitted to the interpreter is being passed through the mustache render engine, so all valid mustache sequences are expanded with appropriate variable values.

Public Types

using command_type = std::function<std::string(const command_statement&)>#

Type of the command handler used to process command calls.

Interpreter executes a registered command, when an LLM model requests for an execution.

Public Functions

void declare_command(const std::string &declaration, command_type command)#

Declare the command available for execution.

The declaration format depends on the underlying command scanner. By default command scanner is a json_command_scanner, and declaration should be a JSON Schema of the command and it’s parameters.

All command declarations are appended to the variable {{metalchat_commands}}.

auto command = R"({
"name":"multiply",
"type": "function",
"description":"multiply two numbers",
"parameters":{
  "a":{"type":"number","description":"first number"},
  "b":{"type":"number","description":"second number"}
}})";

interpreter interp(/* ... *&zwj;/);
interp.declare_command(command, [](const command_statement&) -> std::string {
   return R"({"result": nan})";
});

Parameters:

  • declaration – A command declaration (i.e. JSON Schema by default).

  • command – A handler that returns the result of command execution.

void declare_variable(const std::string &declaration, const std::string &value)#

Declare the variable.

The variable should not start with $-expansion symbol.

interpreter interp(/* ... *&zwj;/);
interp.declare_variable("my_var", R"(arbitrary text)");
Parameters:

  • declaration – A variable name.

  • value – A variable value.

class basic_message#

Repository#

template<language_transformer Transformer, typename Document = safetensor_document>
struct filesystem_repository#

A filesystem-based read-only repository used to retrieve language transformer building blocks (layer options, layer, and string tokenizer).

Template Parameters:

  • Transformer – a transformer specification.

  • Document – a document format type.

template<language_transformer Transformer, readonly_filesystem FileSystem>
struct huggingface_repository#

A repository that dynamically retrieves transformers from HuggingFace repository.

The implementation does not assume transport used to access HuggingFace repository, therefore users must provide a necessary implementation and authentication of requests.

Template Parameters:

  • Transformer – transformer specification.

  • FileSystem – a read-only file access system used to download the transformer.

Transformer#

class basic_transformer#

Subclassed by metalchat::transformer_wrapper< Transformer >

template<typename Layer>
class transformer#

Public Functions

inline void set_sampler(const sampler_pointer &sampler) noexcept#

Replace current sampler with the specified implementation.

When a null pointer is provided, then the sampler is set to an empty sequential sampler, which means that the transform method returns all vocabulary indices.

Parameters:

sampler – a new sampler instance to use in the transformation.

template<immutable_tensor2_t<index_type> Input>
inline auto transform(Input input, std::size_t start_pos = 0)#

Transform the input sequence of tokens to the most probable token.

Parameters:

  • input – an input sequence of token identifiers (from a model vocabulary).

  • start_pos – a start position to generate a token from.