Support for B200 (`sm_100` with `pytorch>=2.7.0`)

[dominiquegarmier]

Is there already an open issue addressing/discussing #37760 (I could not find any)? Looks like there is an issue open on the pytorch repo pytorch/pytorch#152275

Feel free to close if this is a duplicate.

[anijain2305]

@gante Is it possible to try out this PR from your side?

pytorch/pytorch#152287

One way to try this would be - get torch 2.7 and then manually apply the changes to the relevant files. I provided more details on why the slowdown here - pytorch/pytorch#152275. In my local testing, the latency regression was mitigated. But, we would need HF team to confirm.

[gante]

@anijain2305 As replied in pytorch/pytorch#152287 -- I can confirm this torch PR solves the slowdown issue. After we have a torch release with this change, we can unpin transformers ✅

Meanwhile @dominiquegarmier, if you're not using torch.compile with gemma3-like models, you can manually install torch>=2.7.0 after installing transformers. The torch<2.7.0 pin was added to prevent speed regressions with a popular model :)

[dominiquegarmier]

Thanks, we have a workaround atm where we compile multiple libaries from source.

I mainly opened the issue because many of our libraries we use depend in some way on transformers. Basically, I think many other libraries are holding out on upgrading until transformers does.

[KyleMylonakisProtopia]

@anijain2305 As replied in pytorch/pytorch#152287 -- I can confirm this torch PR solves the slowdown issue. After we have a torch release with this change, we can unpin transformers ✅

Meanwhile @dominiquegarmier, if you're not using torch.compile with gemma3-like models, you can manually install torch>=2.7.0 after installing transformers. The torch<2.7.0 pin was added to prevent speed regressions with a popular model :)

It looks like this release has happened (https://.com/pytorch/pytorch/releases/tag/v2.7.1). Is there a transformers release planned which remove this pin?

[KyleMylonakisProtopia]

I went ahead and opened a PR which removes the pin.

#38789