Summary:
Add last_token_pos in the forward options.

Purpose:

• the last norm and output of lm-head can be performed with the last valid token at prefill.
• If the input sequence length is fixed when an accelerator doesn't support the dynamic shapes, selecting the last token from the input is not always guaranteed as valid.
• Thus, it needs an additional pointer to select the last valid token only to perform the last norm and output.

Differential Revision: D76440105

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11793

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 4 New Failures, 1 Unrelated Failure

As of commit b526c43 with merge base 3c05b6c ():

NEW FAILURES - The following jobs have failed:

• Build Linux Wheels / pytorch/executorch / build-manywheel-py3_10-cpu (gh)
  ModuleNotFoundError: No module named 'torchvision'
• Build Linux Wheels / pytorch/executorch / upload / upload-manywheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_x86_64
• Build macOS Wheels / pytorch/executorch / build-wheel-py3_10-cpu (gh)
  ModuleNotFoundError: No module named 'torchvision'
• Build macOS Wheels / pytorch/executorch / upload / upload-wheel-py3_10-cpu (gh)
  Unable to download artifact(s): Artifact not found for name: pytorch_executorch__3.10_cpu_

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / test-moshi-linux / linux-job (gh) (trunk failure)
  test_exported_decoder_xnnpack

This comment was automatically generated by Dr. CI and updates every 15 minutes.

This pull request was exported from Phabricator. Differential Revision: D76440105

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorct, for example
@pytorct label "release notes: none"

For more information, see
https://.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.