|
TT Client Library 1.0.0
C++ library for Tandem Tales clients.
|
Represents the external API used for text embedding and completion. More...
#include <tt/ExternalAPI.h>
Public Member Functions | |
| ExternalAPI (const std::string &key) | |
| Constructs an ExternalAPI object. | |
| ExternalAPI () | |
| Constructs an ExternalAPI object. | |
| std::vector< float > | embed (std::string text, int dim=1024) |
| Queries the external text embedding API and returns its response as a vector of floats. | |
| std::string | complete (const std::vector< std::string > &messages) |
| Queries the external text completion API and returns its response as a string. | |
| std::string | complete (const std::string &message) |
| Queries the external text completion API and returns its response as a string. | |
| void | setReasoning (const std::optional< std::string > &value=std::nullopt) |
| Sets the reasoning level used by the large language model during text completion. | |
| void | setMaxT (const std::optional< int > &value=std::nullopt) |
| Sets the maximum number of tokens the model may use to generate its response. | |
| void | setTemp (const std::optional< float > &value=std::nullopt) |
| Sets the temperature, which influences the randomness of the response. | |
| void | setTopK (const std::optional< int > &value=std::nullopt) |
| When choosing a next token to add to the output, the model may only choose from the best k tokens. | |
| void | setTopP (const std::optional< float > &value=std::nullopt) |
| When choosing a next token to add to the output, the model may only choose from tokens whose probabilities add to p. | |
| void | setMinP (const std::optional< float > &value=std::nullopt) |
| When choosing a next token to add to the output, the model may only choose from tokens whose probability is p times the probability of the most likely token. | |
| void | setKey (const std::string &k) |
| Sets the API key. | |
| const std::optional< std::string > & | getReasoning () const |
| Gives the current reasoning value. | |
| const std::optional< int > & | getMaxT () const |
| Gives the current maxt value. | |
| const std::optional< float > & | getTemp () const |
| Gives the current temperature value. | |
| const std::optional< int > & | getTopK () const |
| Gives the current topk value. | |
| const std::optional< float > & | getTopP () const |
| Gives the current topp value. | |
| const std::optional< float > & | getMinP () const |
| Gives the current minp value. | |
| const std::string & | getKey () const |
| Gives the API key. | |
| const int | getRemainingInTokens () const |
| Gives the remaining amount of in tokens for completion. | |
| const int | getRemainingOutTokens () const |
| Gives the remaining amount of out tokens for completion. | |
| const int | getRemainingEmbedTokens () const |
| Gives the remaining amount of tokens for embed. | |
| const int | getInTokensUsed () const |
| Gives the amount of in tokens used for completion. | |
| const int | getOutTokensUsed () const |
| Gives the amount of out tokens used for completion. | |
| const int | getEmbedTokensUsed () const |
| Gives the amount of tokens used for embed. | |
Represents the external API used for text embedding and completion.
| tt::ExternalAPI::ExternalAPI | ( | const std::string & | key | ) |
Constructs an ExternalAPI object.
| key | the api key to use |
| std::string tt::ExternalAPI::complete | ( | const std::string & | message | ) |
Queries the external text completion API and returns its response as a string.
With a single string as input, the model will use its default system prompt and treat the string as a message from the user.
Each time this method is called, the number of tokens used in the prompt is added to complete_in_tokens and the number of tokens used in reasoning and completion is added to complete_out_tokens. If the number of tokens used exceeds COMPLETE_IN_TOKENS_LIMIT or COMPLETE_OUT_TOKENS_LIMIT respectively, this method will raise an exception.
| message | the message from the user. |
| std::logic_error | if the client does not have an API key |
| std::string tt::ExternalAPI::complete | ( | const std::vector< std::string > & | messages | ) |
Queries the external text completion API and returns its response as a string.
Each time this method is called, the number of tokens used in the prompt is added to completeInTokens and the number of tokens used in reasoning and completion is added to completeOutTokens. If the number of tokens used exceeds COMPLETE_IN_TOKENS_LIMIT or COMPLETE_OUT_TOKENS_LIMIT respectively, this method will raise an exception.
| messages | An array of strings representing a conversation. The first string will be used as the system prompt. Every odd numbered message will be treated as a message from the user. Every even numbered message (after the first) will be treated as a response from the assistant. |
| std::logic_error | if the client does not have an API key |
| std::vector< float > tt::ExternalAPI::embed | ( | std::string | text, |
| int | dim = 1024 ) |
Queries the external text embedding API and returns its response as a vector of floats.
Each time this method is called, the number of tokens used is added to embedTokens. If the number of tokens used exceeds EMBED_TOKENS_LIMIT, this method will raise an exception.
| text | The text which will be embedded as an array of floats. |
| dim | The number of dimensions in the returned array. |
| const int tt::ExternalAPI::getEmbedTokensUsed | ( | ) | const |
Gives the amount of tokens used for embed.
| const int tt::ExternalAPI::getInTokensUsed | ( | ) | const |
Gives the amount of in tokens used for completion.
| const std::string & tt::ExternalAPI::getKey | ( | ) | const |
Gives the API key.
| const std::optional< int > & tt::ExternalAPI::getMaxT | ( | ) | const |
Gives the current maxt value.
| const std::optional< float > & tt::ExternalAPI::getMinP | ( | ) | const |
Gives the current minp value.
| const int tt::ExternalAPI::getOutTokensUsed | ( | ) | const |
Gives the amount of out tokens used for completion.
| const std::optional< std::string > & tt::ExternalAPI::getReasoning | ( | ) | const |
Gives the current reasoning value.
| const int tt::ExternalAPI::getRemainingEmbedTokens | ( | ) | const |
Gives the remaining amount of tokens for embed.
| const int tt::ExternalAPI::getRemainingInTokens | ( | ) | const |
Gives the remaining amount of in tokens for completion.
| const int tt::ExternalAPI::getRemainingOutTokens | ( | ) | const |
Gives the remaining amount of out tokens for completion.
| const std::optional< float > & tt::ExternalAPI::getTemp | ( | ) | const |
Gives the current temperature value.
| const std::optional< int > & tt::ExternalAPI::getTopK | ( | ) | const |
Gives the current topk value.
| const std::optional< float > & tt::ExternalAPI::getTopP | ( | ) | const |
Gives the current topp value.
| void tt::ExternalAPI::setKey | ( | const std::string & | k | ) |
Sets the API key.
| k | the key |
| void tt::ExternalAPI::setMaxT | ( | const std::optional< int > & | value = std::nullopt | ) |
Sets the maximum number of tokens the model may use to generate its response.
To use the default value, you can pass std::nullopt as a parameter, or pass no parameter at all.
| value | the maxt to use |
| void tt::ExternalAPI::setMinP | ( | const std::optional< float > & | value = std::nullopt | ) |
When choosing a next token to add to the output, the model may only choose from tokens whose probability is p times the probability of the most likely token.
To use the default value, you can pass std::nullopt as a parameter, or pass no parameter at all.
| value | the minp to use |
| void tt::ExternalAPI::setReasoning | ( | const std::optional< std::string > & | value = std::nullopt | ) |
Sets the reasoning level used by the large language model during text completion.
Value includes max, high, medium, low, and none
To use the default value, you can pass std::nullopt as a parameter, or pass no parameter at all.
| value | the reasoning level to use |
| void tt::ExternalAPI::setTemp | ( | const std::optional< float > & | value = std::nullopt | ) |
Sets the temperature, which influences the randomness of the response.
A value of 0.0 will give deterministic responses. The max value is 2.0.
To use the default value, you can pass std::nullopt as a parameter, or pass no parameter at all.
| value | the temperature to use |
| void tt::ExternalAPI::setTopK | ( | const std::optional< int > & | value = std::nullopt | ) |
When choosing a next token to add to the output, the model may only choose from the best k tokens.
To use the default value, you can pass std::nullopt as a parameter, or pass no parameter at all.
| value | the topk to use |
| void tt::ExternalAPI::setTopP | ( | const std::optional< float > & | value = std::nullopt | ) |
When choosing a next token to add to the output, the model may only choose from tokens whose probabilities add to p.
To use the default value, you can pass std::nullopt as a parameter, or pass no parameter at all.
| value | the topp to use |