U Ph1@sddlmZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z GdddejZGdddeZGd d d ejZGd d d eZGd ddeZGdddeZdS)) annotationsN) Convolution)ActConvNormPool split_argsc sTeZdZdZddddifddfdddd d d d d fd d ZdddddZZS)ChannelSELayerz Re-implementation of the Squeeze-and-Excitation block based on: "Hu et al., Squeeze-and-Excitation Networks, https://arxiv.org/abs/1709.01507". reluinplaceTsigmoidFinttuple[str, dict] | strboolNone) spatial_dims in_channelsr acti_type_1 acti_type_2 add_residualreturnc st||_ttj|f}|d|_t||}|dkrRtd|d|dt|\} } t|\} } t t j ||ddt | f| t j ||ddt | f| |_ dS) aS Args: spatial_dims: number of spatial dimensions, could be 1, 2, or 3. in_channels: number of input channels. r: the reduction ratio r in the paper. Defaults to 2. acti_type_1: activation type of the hidden squeeze layer. Defaults to ``("relu", {"inplace": True})``. acti_type_2: activation type of the output squeeze layer. Defaults to "sigmoid". Raises: ValueError: When ``r`` is nonpositive or larger than ``in_channels``. See also: :py:class:`monai.networks.layers.Act` rz7r must be positive and smaller than in_channels, got r=z in_channels=.T)biasN)super__init__rr ADAPTIVEAVGavg_poolr ValueErrorrnn SequentialLinearrfc) selfrrrrrr pool_typechannelsZact_1Z act_1_argsZact_2Z act_2_args __class__a/home/dell461/cl/sdc2/HISourceFinder-master-l/src/monai/networks/blocks/squeeze_and_excitation.pyrs       zChannelSELayer.__init__ torch.TensorxrcCsb|jdd\}}||||}||||gdg|jd}||}|jr^||7}|S)b Args: x: in shape (batch, in_channels, spatial_1[, spatial_2, ...]). Nr r)shaperviewr$ndimr)r%r.bcyresultr*r*r+forwardJs$zChannelSELayer.forward__name__ __module__ __qualname____doc__rr7 __classcell__r*r*r(r+r s   -r cs2eZdZdZd dddddddfd d ZZS) ResidualSELayerz A "squeeze-and-excitation"-like layer with a residual connection:: --+-- SE --o-- | | +--------+ r leakyrelur rrr)rrrrrrcstj|||||dddS)a Args: spatial_dims: number of spatial dimensions, could be 1, 2, or 3. in_channels: number of input channels. r: the reduction ratio r in the paper. Defaults to 2. acti_type_1: defaults to "leakyrelu". acti_type_2: defaults to "relu". See also: :py:class:`monai.networks.blocks.ChannelSELayer` T)rrrrrrN)rr)r%rrrrrr(r*r+reszResidualSELayer.__init__)r r?r )r9r:r;r<rr=r*r*r(r+r>\s  r>cspeZdZdZddddddddifddddiffdddddd d d d dd d d d fdd ZdddddZZS)SEBlockac Residual module enhanced with Squeeze-and-Excitation:: ----+- conv1 -- conv2 -- conv3 -- SE -o--- | | +---(channel project if needed)----+ Re-implementation of the SE-Resnet block based on: "Hu et al., Squeeze-and-Excitation Networks, https://arxiv.org/abs/1709.01507". Nr r r Tr rz dict | NoneConvolution | Nonerztuple[str, dict] | str | None) rrn_chns_1n_chns_2n_chns_3 conv_param_1 conv_param_2 conv_param_3projectrrracti_type_finalcs(t|s$dtjdddifd}tf|||d||_|sVdtjdddifd}tf|||d||_|sdtjdd}tf|||d||_t||| | | d |_ | dkr||krt t j |f||dd |_ n| dkrt |_ n| |_ | dk rt| \}}t|f||_n t |_dS) ai Args: spatial_dims: number of spatial dimensions, could be 1, 2, or 3. in_channels: number of input channels. n_chns_1: number of output channels in the 1st convolution. n_chns_2: number of output channels in the 2nd convolution. n_chns_3: number of output channels in the 3rd convolution. conv_param_1: additional parameters to the 1st convolution. Defaults to ``{"kernel_size": 1, "norm": Norm.BATCH, "act": ("relu", {"inplace": True})}`` conv_param_2: additional parameters to the 2nd convolution. Defaults to ``{"kernel_size": 3, "norm": Norm.BATCH, "act": ("relu", {"inplace": True})}`` conv_param_3: additional parameters to the 3rd convolution. Defaults to ``{"kernel_size": 1, "norm": Norm.BATCH, "act": None}`` project: in the case of residual chns and output chns doesn't match, a project (Conv) layer/block is used to adjust the number of chns. In SENET, it is consisted with a Conv layer as well as a Norm layer. Defaults to None (chns are matchable) or a Conv layer with kernel size 1. r: the reduction ratio r in the paper. Defaults to 2. acti_type_1: activation type of the hidden squeeze layer. Defaults to "relu". acti_type_2: activation type of the output squeeze layer. Defaults to "sigmoid". acti_type_final: activation type of the end of the block. Defaults to "relu". See also: :py:class:`monai.networks.blocks.ChannelSELayer` rr r T) kernel_sizenormact)rr out_channelsN)rrrrr)rJ)rrrBATCHrconv1conv2conv3r se_layerrCONVrHr!IdentityrrrL)r%rrrBrCrDrErFrGrHrrrrIZ act_finalZact_final_argsr(r*r+rs@+    zSEBlock.__init__r,r-cCsH||}||}||}||}||}||7}||}|S)r/)rHrPrQrRrSrL)r%r.residualr*r*r+r7s      zSEBlock.forwardr8r*r*r(r+r@s  ,Lr@c s:eZdZdZdZd dddddddddfd d ZZS) SEBottleneckz" Bottleneck for SENet154. rNrrArrinplanesplanesgroups reductionstride downsamplerc sxdddddiftjdd}|ddddiftj|dd} dddtjdd} tj|||d |d |d || | ||d dS) Nrr r TFstridesrJrLrKrrNrarJrLrKr\rr rX rrrBrCrDrErFrGrHrrrOrr r%rrZr[r\r]r^r_rErFrGr(r*r+rs4   zSEBottleneck.__init__)rNr9r:r;r< expansionrr=r*r*r(r+rWs  rWc s:eZdZdZdZd dddddddddfd d ZZS) SEResNetBottleneckz ResNet bottleneck with a Squeeze-and-Excitation module. It follows Caffe implementation and uses `strides=stride` in `conv1` and not in `conv2` (the latter is used in the torchvision implementation of ResNet). rXrNrrArrYc sp|ddddiftjdd}dddddiftj|dd} dddtjdd} tj|||||d || | ||d dS) Nrr r TFr`rNrbrXrcrdrer(r*r+r"s4   zSEResNetBottleneck.__init__)rNrfr*r*r(r+rhs  rhc s<eZdZdZdZd dddddddddd fd d ZZS) SEResNeXtBottleneckzI ResNeXt bottleneck type C with a Squeeze-and-Excitation module. rXrNrrAr) rrZr[r\r]r^r_ base_widthrc sdddddiftjdd} |ddddiftj|dd} dddtjdd} t||d |} tj||| | |d | | | ||d dS) Nrr r TFr`rNrb@rXrc)rrOmathfloorrr) r%rrZr[r\r]r^r_rjrErFrGwidthr(r*r+rRs6   zSEResNeXtBottleneck.__init__)rNrXrfr*r*r(r+riKs  ri) __future__rrltorchtorch.nnr!Zmonai.networks.blocksrmonai.networks.layers.factoriesrrrrrModuler r>r@rWrhrir*r*r*r+ s   E&g02