o ( i1@sddlmZddlZddlZddlmZddlmZddlm Z m Z m Z m Z m Z GdddejZGdddeZGd d d ejZGd d d eZGd ddeZGdddeZdS)) annotationsN) Convolution)ActConvNormPool split_argscs>eZdZdZddddifddfdfdd ZdddZZS)ChannelSELayerz Re-implementation of the Squeeze-and-Excitation block based on: "Hu et al., Squeeze-and-Excitation Networks, https://arxiv.org/abs/1709.01507". reluinplaceTsigmoidF spatial_dimsint in_channelsr acti_type_1tuple[str, dict] | str acti_type_2 add_residualboolreturnNonec st||_ttj|f}|d|_t||}|dkr)td|d|dt|\} } t|\} } t t j ||ddt | d i| t j ||ddt | d i| |_ dS) aS Args: spatial_dims: number of spatial dimensions, could be 1, 2, or 3. in_channels: number of input channels. r: the reduction ratio r in the paper. Defaults to 2. acti_type_1: activation type of the hidden squeeze layer. Defaults to ``("relu", {"inplace": True})``. acti_type_2: activation type of the output squeeze layer. Defaults to "sigmoid". Raises: ValueError: When ``r`` is nonpositive or larger than ``in_channels``. See also: :py:class:`monai.networks.layers.Act` rz7r must be positive and smaller than in_channels, got r=z in_channels=.T)biasN)super__init__rr ADAPTIVEAVGavg_poolr ValueErrorrnn SequentialLinearrfc) selfrrrrrr pool_typechannelsZact_1Z act_1_argsZact_2Z act_2_args __class__rn/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/networks/blocks/squeeze_and_excitation.pyrs      zChannelSELayer.__init__x torch.TensorcCsb|jdd\}}||||}||||gdg|jd}||}|jr/||7}|S)b Args: x: in shape (batch, in_channels, spatial_1[, spatial_2, ...]). Nr r)shaper viewr%ndimr)r&r,bcyresultrrr+forwardJs$zChannelSELayer.forward)rrrrrrrrrrrrrrr,r-rr-__name__ __module__ __qualname____doc__rr6 __classcell__rrr)r+r s -r cs*eZdZdZ   ddfdd ZZS)ResidualSELayerz A "squeeze-and-excitation"-like layer with a residual connection:: --+-- SE --o-- | | +--------+ r leakyrelur rrrrrrrrrcstj|||||dddS)a Args: spatial_dims: number of spatial dimensions, could be 1, 2, or 3. in_channels: number of input channels. r: the reduction ratio r in the paper. Defaults to 2. acti_type_1: defaults to "leakyrelu". acti_type_2: defaults to "relu". See also: :py:class:`monai.networks.blocks.ChannelSELayer` T)rrrrrrN)rr)r&rrrrrr)rr+res zResidualSELayer.__init__)r r?r ) rrrrrrrrrrrr)r9r:r;r<rr=rrr)r+r>\s  r>c sNeZdZdZddddddddifddddiffd!fdd Zd"dd ZZS)#SEBlockac Residual module enhanced with Squeeze-and-Excitation:: ----+- conv1 -- conv2 -- conv3 -- SE -o--- | | +---(channel project if needed)----+ Re-implementation of the SE-Resnet block based on: "Hu et al., Squeeze-and-Excitation Networks, https://arxiv.org/abs/1709.01507". Nr r r Tr rrrn_chns_1n_chns_2n_chns_3 conv_param_1 dict | None conv_param_2 conv_param_3projectConvolution | Nonerrrracti_type_finaltuple[str, dict] | str | Nonecs,t|sdtjdddifd}td |||d||_|s+dtjdddifd}td |||d||_|s@dtjdd}td |||d||_t||| | | d |_ | durl||krlt t j |f||dd |_ n | durvt |_ n| |_ | durt| \}}t|d i||_dSt |_dS) ai Args: spatial_dims: number of spatial dimensions, could be 1, 2, or 3. in_channels: number of input channels. n_chns_1: number of output channels in the 1st convolution. n_chns_2: number of output channels in the 2nd convolution. n_chns_3: number of output channels in the 3rd convolution. conv_param_1: additional parameters to the 1st convolution. Defaults to ``{"kernel_size": 1, "norm": Norm.BATCH, "act": ("relu", {"inplace": True})}`` conv_param_2: additional parameters to the 2nd convolution. Defaults to ``{"kernel_size": 3, "norm": Norm.BATCH, "act": ("relu", {"inplace": True})}`` conv_param_3: additional parameters to the 3rd convolution. Defaults to ``{"kernel_size": 1, "norm": Norm.BATCH, "act": None}`` project: in the case of residual chns and output chns doesn't match, a project (Conv) layer/block is used to adjust the number of chns. In SENET, it is consisted with a Conv layer as well as a Norm layer. Defaults to None (chns are matchable) or a Conv layer with kernel size 1. r: the reduction ratio r in the paper. Defaults to 2. acti_type_1: activation type of the hidden squeeze layer. Defaults to "relu". acti_type_2: activation type of the output squeeze layer. Defaults to "sigmoid". acti_type_final: activation type of the end of the block. Defaults to "relu". See also: :py:class:`monai.networks.blocks.ChannelSELayer` rr r T) kernel_sizenormact)rr out_channelsN)rrrrr)rLr)rrrBATCHrconv1conv2conv3r se_layerrCONVrHr"IdentityrrrN)r&rrrArBrCrDrFrGrHrrrrJZ act_finalZact_final_argsr)rr+rs4 +   zSEBlock.__init__r,r-rcCsH||}||}||}||}||}||7}||}|S)r.)rHrRrSrTrUrN)r&r,residualrrr+r6s      zSEBlock.forward)rrrrrArrBrrCrrDrErFrErGrErHrIrrrrrrrJrKr7r8rrr)r+r@s  Lr@c,eZdZdZdZ  ddfdd ZZS) SEBottleneckz" Bottleneck for SENet154. rNrrinplanesplanesgroups reductionstride downsamplerIrrc sxdddddiftjdd}|ddddiftj|dd} dddtjdd} tj|||d |d |d || | ||d dS) Nrr r TFstridesrLrNrMrrPrcrLrNrMr^rr r[ rrrArBrCrDrFrGrHrrrQrr r&rr\r]r^r_r`rarDrFrGr)rr+rs4    zSEBottleneck.__init__rNrrr\rr]rr^rr_rr`rrarIrrr9r:r;r< expansionrr=rrr)r+rZs  rZcrY)SEResNetBottleneckz ResNet bottleneck with a Squeeze-and-Excitation module. It follows Caffe implementation and uses `strides=stride` in `conv1` and not in `conv2` (the latter is used in the torchvision implementation of ResNet). r[rNrrr\r]r^r_r`rarIrrc sp|ddddiftjdd}dddddiftj|dd} dddtjdd} tj|||||d || | ||d dS) Nrr r TFrbrPrdr[rerfrgr)rr+r"s4    zSEResNetBottleneck.__init__rhrirjrrr)r+rls  rlcs.eZdZdZdZ   ddfdd ZZS)SEResNeXtBottleneckzI ResNeXt bottleneck type C with a Squeeze-and-Excitation module. r[rNrrr\r]r^r_r`rarI base_widthrrc sdddddiftjdd} |ddddiftj|dd} dddtjdd} t||d |} tj||| | |d | | | ||d dS) Nrr r TFrbrPrd@r[re)rrQmathfloorrr) r&rr\r]r^r_r`rarnrDrFrGwidthr)rr+rRs6    zSEResNeXtBottleneck.__init__)rNr[)rrr\rr]rr^rr_rr`rrarIrnrrrrjrrr)r+rmKs rm) __future__rrptorchtorch.nnr"Zmonai.networks.blocksrmonai.networks.layers.factoriesrrrrrModuler r>r@rZrlrmrrrr+s   E&g02