U Ph @sNddlmZddlmZddlmZddlmZddhZGdddej Z dS) ) annotationsN) get_act_layer)look_up_optionvitswincs8eZdZdZddddddd fd d Zd d ZZS)MLPBlockz A multi-layer perceptron block, based on: "Dosovitskiy et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale " GELUrintfloatz tuple | strNone) hidden_sizemlp_dim dropout_rateactreturncstd|krdks(ntd|p.|}|dkrDt||nt||d|_t|||_t||_t ||_ t |t }|dkrt ||_ n |dkr|j |_ ntdt d S) a Args: hidden_size: dimension of hidden layer. mlp_dim: dimension of feedforward layer. If 0, `hidden_size` will be used. dropout_rate: fraction of the input units to drop. act: activation type and arguments. Defaults to GELU. Also supports "GEGLU" and others. dropout_mode: dropout mode, can be "vit" or "swin". "vit" mode uses two dropout instances as implemented in https://github.com/google-research/vision_transformer/blob/main/vit_jax/models.py#L87 "swin" corresponds to one instance as implemented in https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_mlp.py#L23 rz'dropout_rate should be between 0 and 1.GEGLUrrzdropout_mode should be one of N)super__init__ ValueErrornnLinearlinear1linear2rfnDropoutdrop1rSUPPORTED_DROPOUT_MODEdrop2)selfr rrrZ dropout_modeZ dropout_opt __class__N/home/dell461/cl/sdc2/HISourceFinder-master-l/src/monai/networks/blocks/mlp.pyrs &    zMLPBlock.__init__cCs2|||}||}||}||}|S)N)rrrrr )r!xr$r$r%forward?s    zMLPBlock.forward)rr r)__name__ __module__ __qualname____doc__rr' __classcell__r$r$r"r%rs #r) __future__rtorch.nnrmonai.networks.layersr monai.utilsrrModulerr$r$r$r% s