o ( i @sfddlmZddlmZddlmZddlmZddlm Z ddl m Z hdZ Gdd d ej ZdS) ) annotations)UnionN) get_act_layer) split_args)look_up_option>swinvista3dvitcs.eZdZdZ ddfdd ZddZZS)MLPBlockz A multi-layer perceptron block, based on: "Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale " GELUr hidden_sizeintmlp_dim dropout_ratefloatact tuple | strreturnNonec std|krdkstdtd|p|}t|\}}|dkr+t||nt||d|_t|||_t||_ ||t |t }|dkr[t ||_ t ||_d S|dkrkt ||_ |j |_d S|dkr{t|_ t|_d Std t ) a  Args: hidden_size: dimension of hidden layer. mlp_dim: dimension of feedforward layer. If 0, `hidden_size` will be used. dropout_rate: fraction of the input units to drop. act: activation type and arguments. Defaults to GELU. Also supports "GEGLU" and others. dropout_mode: dropout mode, can be "vit" or "swin". "vit" mode uses two dropout instances as implemented in https://github.com/google-research/vision_transformer/blob/main/vit_jax/models.py#L87 "swin" corresponds to one instance as implemented in https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_mlp.py#L23 "vista3d" mode does not use dropout. rz'dropout_rate should be between 0 and 1.GEGLUr rrzdropout_mode should be one of N)super__init__ ValueErrorrnnLinearlinear1linear2rfnrSUPPORTED_DROPOUT_MODEDropoutdrop1drop2Identity) selfr rrrZ dropout_modeact_name_Z dropout_opt __class__[/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/networks/blocks/mlp.pyrs.  &      zMLPBlock.__init__cCs2|||}||}||}||}|S)N)r rr#rr$)r&xr+r+r,forwardKs    zMLPBlock.forward)r r r ) r rrrrrrrrr)__name__ __module__ __qualname____doc__rr. __classcell__r+r+r)r,r s ,r ) __future__rtypingrtorch.nnrmonai.networks.layersrmonai.networks.layers.factoriesr monai.utilsrr!Moduler r+r+r+r,s