Yep that's right! A very common use-case is for "document masking" (i.e. variable length sequences), and that requires recomputing the mask on every iteration (which isn't "free", but is on the order of microseconds to milliseconds and not seconds).
Horace He
@chhillee.bsky.social
833 followers 60 following 13 posts
@PyTorch "My learning style is Horace twitter threads" - @typedfemale
