Represents a language model for making chat requests.
@link lm.selectChatModels}
See also:
{
Fields
read onlyversion:String
Opaque version string of the model. This is defined by the extension contributing the language model and subject to change.
read onlyvendor:String
A well-known identifier of the vendor of the language model. An example is copilot
, but
values are defined by extensions contributing chat models and need to be looked up with them.
sendRequest(messages:Array<LanguageModelChatMessage>, ?options:LanguageModelChatRequestOptions, ?token:CancellationToken):Thenable<LanguageModelChatResponse>
Make a chat request using a language model.
Note that language model use may be subject to access restrictions and user consent. Calling this function for the first time (for an extension) will show a consent dialog to the user and because of that this function must only be called in response to a user action! Extensions can use {@link LanguageModelAccessInformation.canSendRequest} to check if they have the necessary permissions to make a request.
This function will return a rejected promise if making a request to the language model is not possible. Reasons for this can be:
- user consent not given, see {@link LanguageModelError.NoPermissions
NoPermissions
} - model does not exist anymore, see {@link LanguageModelError.NotFound
NotFound
} - quota limits exceeded, see {@link LanguageModelError.Blocked
Blocked
} - other issues in which case extension must check {@link LanguageModelError.cause
LanguageModelError.cause
}
An extension can make use of language model tool calling by passing a set of tools to {@link LanguageModelChatRequestOptions.tools}. The language model will return a {@link LanguageModelToolCallPart} and the extension can invoke the tool and make another request with the result.
@link CancellationTokenSource} for how to create one. @link LanguageModelChatResponse}. The promise will reject when the request couldn't be made.
Parameters:
messages | An array of message instances. |
---|---|
options | Options that control the request. |
token | A cancellation token which controls the request. See { |
Returns:
A thenable that resolves to a {
read onlymaxInputTokens:Int
The maximum number of tokens that can be sent to the model in a single request.
read onlyfamily:String
Opaque family-name of the language model. Values might be gpt-3.5-turbo
, gpt4
, phi2
, or llama
but they are defined by extensions contributing languages and subject to change.
countTokens(text:EitherType<String, LanguageModelChatMessage>, ?token:CancellationToken):Thenable<Int>
Count the number of tokens in a message using the model specific tokenizer-logic.
@link CancellationTokenSource} for how to create one.
Parameters:
text | A string or a message instance. |
---|---|
token | Optional cancellation token. See { |
Returns:
A thenable that resolves to the number of tokens.