AI生成Minecraft(open-oasis) 代码浅读 - 1(Attention and Embedding)

type

status

date

slug

summary

VIT-VAE

1. Attention

这里用了两种 TemporalAxialAttention (Time dim in sequence of video)

和 SpatialAxialAttention

D → h(heads) * d

The attention mechanism is applied along the temporal axis (T), meaning that the model learns how the information at different time steps relates to each other

两种 rotary_emb 使用方法不同

rotary_emb module is used to embed the queries (q) and keys (k) to incorporate the frequency information, which is helpful in capturing periodic or sequential dependencies.

is_causal == True: Only attend to past time steps

Spatial —> Attention computed on H, W

dependencies between different spatial locations

is_causal == False: Attend to the whole image

推荐: diffusion-forcing 的作者这里的rotary_emb attention代码来源

‣

关键词：生成式3d