package ppl.llm.proto

Get desktop application:
View/edit binary Protocol Buffers messages

• rpc Generation (BatchedRequest, stream BatchedResponse)

  llm.proto:6

  message BatchedRequest

  llm.proto:55

  • repeated Request req = 1

  message BatchedResponse

  llm.proto:85

  • repeated Response rsp = 1

Used in: Response

• float logprobs = 1

• bool is_special = 2

• FinishReason finish_reason = 3

Used in: Detail

• FINISH_REASON_LENGTH = 0

• FINISH_REASON_EOS_TOKEN = 1

• FINISH_REASON_STOP_SEQUENCE = 2

Used in: Request

• float temperature = 1

  / exponential scaling output probability distribution

• uint32 top_k = 2

  / restricting to the k highest probability elements

• float top_p = 3

  / restricting to top tokens summing to prob_cut_off <= prob_cut_off

• float typical_p = 4

  / [not used] restricting to top tokens summing to prob_cut_off <= prob_cut_off, not used

• bool do_sample = 5

  / apply sampling on the logits

• uint64 seed = 6

  / [not used] random seed for sampling, not used

• float repetition_penalty = 7

  / repetition penalty

• float presence_penalty = 8

  / repetition penalty

• float frequency_penalty = 9

  / repetition penalty

• bool watermark = 10

  / [not used] token watermarking using "A Watermark for Large Language Models"

Used in: BatchedRequest

• uint64 id = 1

• string prompt = 2

• optional Tokens tokens = 3

• optional NextTokenChooserParameters choosing_parameters = 4

• optional StoppingCriteriaParameters stopping_parameters = 5

Used in: BatchedResponse

• Status status = 1

• uint64 id = 2

• string generated = 3

• optional Tokens tokens = 4

• optional Detail detail = 5

Used in: Response

• PROCESSING = 0

• FINISHED = 1

• FAILED = 2

Used in: Request

• uint32 max_new_tokens = 1

  / Maximum number of generated tokens

• optional Tokens stop_tokens = 2

  / Optional stopping tokens array

• bool ignore_eos_token = 3

  / Ignore end of sequence token / used for benchmarking

Used in: Request, Response, StoppingCriteriaParameters

• repeated uint32 ids = 1

  / Token IDs