Merged

docs: fix the Palm2TextGenerator output token size #649

Changes from all commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Failed to load files.
docs: fix the Palm2TextGenerator output token size
  • Loading branch information
@ashleyxuu
ashleyxuu committedMay 1, 2024
commit c1d346c79d00b5e222f182acf7ccb3b61df3e274
Original file line numberDiff line numberDiff line change
Expand Up@@ -233,7 +233,7 @@ def predict(
max_output_tokens (int, default 128):
Maximum number of tokens that can be generated in the response. Specify a lower value for shorter responses and a higher value for longer responses.
A token may be smaller than a word. A token is approximately four characters. 100 tokens correspond to roughly 60-80 words.
Default 128. For the 'text-bison' model, possible values are in the range [1, 1024]. For the 'text-bison-32k' model, possible values are in the range [1, 8196].
Default 128. For the 'text-bison' model, possible values are in the range [1, 1024]. For the 'text-bison-32k' model, possible values are in the range [1, 8192].
Please ensure that the specified value for max_output_tokens is within the appropriate range for the model being used.

top_k (int, default 40):
Expand DownExpand Up@@ -269,10 +269,10 @@ def predict(

if (
self.model_name == _TEXT_GENERATOR_BISON_32K_ENDPOINT
and max_output_tokens not in range(1, 8197)
and max_output_tokens not in range(1, 8193)
):
raise ValueError(
f"max_output_token must be [1, 8196] for TextBison 32k model, but is {max_output_tokens}."
f"max_output_token must be [1, 8192] for TextBison 32k model, but is {max_output_tokens}."
)

if top_k not in range(1, 41):
Expand Down