Class LogitBias

Namespace

FoundationaLLM.Common.Models.Orchestration.Direct

Assembly

FoundationaLLM.Common.dll

Modify the likelihood of specified tokens appearing in the completion. Accepts a json object that maps tokens (specified by their token ID in the GPT tokenizer) to an associated bias value from -100 to 100. You can use this tokenizer tool (which works for both GPT-2 and GPT-3) to convert text to token IDs. Mathematically, the bias is added to the logits generated by the model prior to sampling. The exact effect varies per model, but values between -1 and 1 should decrease or increase likelihood of selection; values like -100 or 100 should result in a ban or exclusive selection of the relevant token. As an example, you can pass {"50256": -100} to prevent the endoftext token from being generated.

public class LogitBias

Inheritance

object

LogitBias

Inherited Members

object.Equals(object)

object.Equals(object, object)

object.GetHashCode()

object.GetType()

object.MemberwiseClone()

object.ReferenceEquals(object, object)

object.ToString()

Extension Methods

ObjectExtensions.GetPropertyValues(object, List<string>)

Properties

Bias

The bias value to apply to the token.

public float? Bias { get; set; }

Property Value

float?

TokenId

The token ID of which to modify the likelihood.

public string? TokenId { get; set; }

Property Value

string