i find the decoder which in your implementation is conv2d,and it is different with MPLDecoder which used in segformer paper ?

[AncientRemember]

did you test performance and infer speed,are conv2d is better than mlpdecode?

[AncientRemember]

the MixFeedForward is alse only used conv2d @ lucidrains

[lucidrains]

a conv2d of kernel size of 1 is equivalent to a linear layer. I did it to avoid transposes

[AncientRemember]

a conv2d of kernel size of 1 is pointwise conv,and it's parameters reused for all point, did i make a mistake?

[lucidrains]

[lucidrains]

a pointwise conv is a linear layer applied on the channel axis (dimension 1)

[AncientRemember]

maybe use the pointwise conv to change feature channels is faster,but i think the grouped conv with groups=channels and kernelsize = HxW equal to linear

[lucidrains]

I don't quite follow, I don't think they are projecting along the sequence dimension (height x width)