Review deck

Mask Data Stream And PyTorch Review

Recall where src_mask and tgt_mask flow through encoder, decoder self-attention, decoder cross-attention, and PyTorch APIs.

All decks
Mask Data Stream And PyTorch Review reference figure
Review the mask data stream until encoder, decoder self-attention, and decoder cross-attention each get the right mask.
question
answer

Q1: Which mask does the encoder use in the Harvard EncoderDecoder flow?

encodersrc-mask

Q2: Which mask is used by decoder self-attention?

decodertgt-mask

Q3: Which mask is used by decoder cross-attention in the Harvard flow?

cross-attentionsrc-mask

Q4: How do src_mask and tgt_mask differ in shape in the lesson's data-stream explanation?

shapebroadcasting

Q5: How does PyTorch distinguish attention masks from key-padding masks?

pytorchmask-types
Press any key