o ) i@sddlmZddlZddlmZddlZddlZddlm Z ddl m m Z ddl mmZddlmZddlmZddlmZmZmZmZddlmZmZddlmZmZmZed d d \ZZ gd Z!Gd dde j"Z#ddZ$ddZ%d&ddZ&Gddde j"Z'Gddde j"Z(Gddde j"Z)Gddde)Z*e*e)dZ+ddZ,Gd d!d!e j"Z-Gd"d#d#e j"Z.d$d%Z/dS)') annotationsN)Sequence) LayerNorm)MLPBlock) PatchEmbed UnetOutBlockUnetrBasicBlock UnetrUpBlock)DropPath trunc_normal_)ensure_tuple_replook_up_optionoptional_importeinops rearrange)name) SwinUNETRwindow_partitionwindow_reverseWindowAttentionSwinTransformerBlock PatchMergingPatchMergingV2 MERGING_MODE BasicLayerSwinTransformercsjeZdZdZdddddddd d d d dejd d d d d fd4fd,d- Zd.d/Zej j d0d1Z d2d3Z Z S)5rz Swin UNETR based on: "Hatamizadeh et al., Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images " )rrrr) T@r instanceFrmerging in_channelsint out_channels patch_sizedepths Sequence[int] num_heads window_sizeSequence[int] | intqkv_biasbool mlp_ratiofloat feature_size norm_name tuple | str drop_rateattn_drop_ratedropout_path_rate normalize norm_layertype[LayerNorm] patch_normuse_checkpoint spatial_dims downsamplestr | nn.Moduleuse_v2returnNonec st|dvr td||_t|j|}t||}d| kr(dks-tdtdd| kr:dks?tdtdd| krLdksQtdtd| ddkr[td ||_td%id |d | d |d |d|d|d|d|d| d| d| d|d|d|d|dt|trt |t n|d||_ t ||| dd| dd|_ t || | dd| dd|_t |d| d| dd| dd|_t |d| d| dd| dd|_t |d | d | dd| dd|_t|d | d!| dd| dd"|_t|| d!| ddd| dd"|_t|| d| ddd| dd"|_t|| d| dd| dd"|_t|| | dd| dd"|_t|| |d#|_d$Sd||_ t ||| dd| dd|_ t || | dd| dd|_t |d| d| dd| dd|_t |d| d| dd| dd|_t |d | d | dd| dd|_t|d | d!| dd| dd"|_t|| d!| ddd| dd"|_t|| d| ddd| dd"|_t|| d| dd| dd"|_t|| | dd| dd"|_t|| |d#|_d$S)&aH Args: in_channels: dimension of input channels. out_channels: dimension of output channels. patch_size: size of the patch token. feature_size: dimension of network feature size. depths: number of layers in each stage. num_heads: number of attention heads. window_size: local window size. qkv_bias: add a learnable bias to query, key, value. mlp_ratio: ratio of mlp hidden dim to embedding dim. norm_name: feature normalization type and arguments. drop_rate: dropout rate. attn_drop_rate: attention dropout rate. dropout_path_rate: drop path rate. normalize: normalize output intermediate features in each stage. norm_layer: normalization layer. patch_norm: whether to apply normalization to the patch embedding. Default is False. use_checkpoint: use gradient checkpointing for reduced memory usage. spatial_dims: number of spatial dims. downsample: module used for downsampling, available options are `"mergingv2"`, `"merging"` and a user-specified `nn.Module` following the API defined in :py:class:`monai.networks.nets.PatchMerging`. The default is currently `"merging"` (the original version defined in v0.9.0). use_v2: using swinunetr_v2, which adds a residual convolution block at the beggining of each swin stage. Examples:: # for 3D single channel input with size (96,96,96), 4-channel output and feature size of 48. >>> net = SwinUNETR(in_channels=1, out_channels=4, feature_size=48) # for 3D 4-channel input with size (128,128,128), 3-channel output and (2,4,2,2) layers in each stage. >>> net = SwinUNETR(in_channels=4, out_channels=3, depths=(2,4,2,2)) # for 2D single channel input with size (96,96), 2-channel output and gradient checkpointing. >>> net = SwinUNETR(in_channels=3, out_channels=2, use_checkpoint=True, spatial_dims=2) )rrz#spatial dimension should be 2 or 3.rz'dropout rate should be between 0 and 1.z1attention dropout rate should be between 0 and 1.z)drop path rate should be between 0 and 1.rz'feature_size should be divisible by 12.in_chans embed_dimr-r)r*r,r1r/r6r7drop_path_rater:r<r=r>r?rArTr>r&r( kernel_sizestrider4 res_blockr)r>r&r(rIupsample_kernel_sizer4rK)r>r&r(N)super__init__ ValueErrorr)r r9r isinstancestrr rswinViTrencoder1encoder2encoder3encoder4 encoder10r decoder5decoder4decoder3decoder2decoder1rout)selfr&r(r)r*r,r-r/r1r3r4r6r7r8r9r:r<r=r>r?rAZ patch_sizes __class__rP`/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/networks/nets/swin_unetr.pyrR4s =                            zSwinUNETR.__init__c CsN|jjd}|jjd}|jjd}|jjd}|d}t|jjjj |d|jjjj |d|j D] \}}|j||ddq<|jdurl|j} | jj |d| jj |d| jj |d |j D] \}}|j||d dqq|jdur|j} | jj |d | jj |d | jj |d |j D] \}}|j||ddq|jdur|j} | jj |d| jj |d| jj |d|j D] \}}|j||ddq|jdur|j} | jj |d| jj |d| jj |dWddSWddS1s wYdS)Nr state_dictzmodule.patch_embed.proj.weightzmodule.patch_embed.proj.biaslayers1)n_blocklayerz,module.layers1.0.downsample.reduction.weightz'module.layers1.0.downsample.norm.weightz%module.layers1.0.downsample.norm.biaslayers2z,module.layers2.0.downsample.reduction.weightz'module.layers2.0.downsample.norm.weightz%module.layers2.0.downsample.norm.biaslayers3z,module.layers3.0.downsample.reduction.weightz'module.layers3.0.downsample.norm.weightz%module.layers3.0.downsample.norm.biaslayers4z,module.layers4.0.downsample.reduction.weightz'module.layers4.0.downsample.norm.weightz%module.layers4.0.downsample.norm.bias)rVrgrjrkrltorchno_grad patch_embedprojweightcopy_biasblocksnamed_children load_fromr? reductionnorm) rbweightsZ layers1_0Z layers2_0Z layers3_0Z layers4_0ZwstateZbnameblockdrPrPrervsN         !$zSwinUNETR.load_fromcCs`t|}|t|jddk}|r.t|dd}td|d|d|jddS)Nrrzspatial dimensions z of input image (spatial shape: z) must be divisible by z**5.)nparraypowerr)anywheretolistrS)rb spatial_shapeimg_size remainderZ wrong_dimsrPrPre_check_input_size0s zSwinUNETR._check_input_sizecCstjstjs||jdd|||j}||}| |d}| |d}| |d}| |d}| ||d}|||} || |} || |} || |} || } | S)NrrrDrLr)rmjit is_scripting is_tracingrshaperVr9rWrXrYrZr[r\r]r^r_r`ra)rbx_inZhidden_states_outZenc0Zenc1Zenc2Zenc3Zdec4dec3dec2dec1dec0ralogitsrPrPreforward;s      zSwinUNETR.forward)*r&r'r(r'r)r'r*r+r,r+r-r.r/r0r1r2r3r'r4r5r6r2r7r2r8r2r9r0r:r;r<r0r=r0r>r'r?r@rAr0rBrC)__name__ __module__ __qualname____doc__nnrrRrvrmrunusedrr __classcell__rPrPrcrer-s4 O.  rc Cs|}t|dkrN|\}}}}}||||d|d||d|d||d|d|}|ddddddddd |d|d|d|}|S|j\}}}}||||d|d||d|d|}|ddddddd |d|d|}|S) a)window partition operation based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer Args: x: input tensor. window_size: local window size. r|rrDrrrLrr!)sizelenviewpermute contiguousr) xr-x_shapebr{hwcwindowsrPrPrerMs(    8 ,.rc Cst|dkrA|\}}}}||||d||d||d|d|d|dd}|dddddddd ||||d}|St|dkrt|\}}}||||d||d|d|dd}|dddddd|||d}|S) aOwindow reverse operation based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer Args: windows: windows tensor. window_size: local window size. dims: dimension values. rLrrDrrr|rrr!)rrrr)rr-dimsrr{rrrrPrPreros&    *  ,$rcCsvt|}|dur t|}tt|D]}||||kr*||||<|dur*d||<q|dur3t|St|t|fS)aQComputing window size based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer Args: x_size: input size. window_size: local window size. shift_size: window shifting size. Nr)listrangertuple)x_sizer- shift_sizeZuse_window_sizeZuse_shift_sizeirPrPreget_window_sizes  rcs2eZdZdZ   ddfdd ZddZZS)ra Window based multi-head self attention module with relative position bias based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer Fr$dimr'r,r-r+r/r0 attn_dropr2 proj_droprBrCcst||_||_||_||}|d|_tjj}t |jdkrt t d|jddd|jddd|jdd||_ t|jd} t|jd} t|jd} |durottj| | | dd} n tt| | | } t| d} | dddddf| dddddf}|ddd}|dddddf|jdd7<|dddddf|jdd7<|dddddf|jdd7<|dddddfd|jddd|jdd9<|dddddfd|jdd9<nt |jdkrt t d|ddd|dd||_ t|jd} t|jd} |durQttj| | dd} n tt| | } t| d} | dddddf| dddddf}|ddd}|dddddf|jdd7<|dddddf|jdd7<|dddddfd|jdd9<|d }|d |t j||d|d |_t ||_t |||_t ||_t|j d d t jd d|_dS)aA Args: dim: number of feature channels. num_heads: number of attention heads. window_size: local window size. qkv_bias: add a learnable bias to query, key, value. attn_drop: attention dropout rate. proj_drop: dropout rate of output. grrrrDNij)indexingrrelative_position_indexrsg{Gz?)std)r)rQrRrr-r,scalermmeshgrid__kwdefaults__rr Parameterzerosrelative_position_bias_tablearangestackflattenrrsumregister_bufferLinearqkvDropoutrrprr Softmaxsoftmax)rbrr,r-r/rrhead_dimZ mesh_argsZcoords_dcoords_hcoords_wcoordscoords_flattenrelative_coordsrrcrPrerRsf  4 ,(((>.&  ,((,    zWindowAttention.__init__c Csf|j\}}}||||d|j||jddddd}|d|d|d}}} ||j}||dd} |j|j d|d|fd||d} | ddd } | | d} |dur|jd} | || | |j||| d d} | d|j||} | | } n| | } || | j} | | dd|||}||}||}|S)NrrrrDrLr)rrreshaper,rr transposerrcloner unsqueezerrrtodtyperpr) rbrmaskrnrrqkvattnrelative_position_biasnwrPrPrers. .   (    zWindowAttention.forward)Fr$r$)rr'r,r'r-r+r/r0rr2rr2rBrC)rrrrrRrrrPrPrcrers KrcsVeZdZdZddddddejdfd%fdd ZddZdd Zd!d"Z d#d$Z Z S)&rz Swin Transformer block based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer r"Tr$GELUFrr'r,r-r+rr1r2r/r0dropr drop_path act_layerrUr:r;r=rBrCc st||_||_||_||_||_| |_| ||_t ||j||||d|_ | dkr0t | nt |_| ||_t||} t|| | |dd|_dS)as Args: dim: number of feature channels. num_heads: number of attention heads. window_size: local window size. shift_size: window shift size. mlp_ratio: ratio of mlp hidden dim to embedding dim. qkv_bias: add a learnable bias to query, key, value. drop: dropout rate. attn_drop: attention dropout rate. drop_path: stochastic depth rate. act_layer: activation layer. norm_layer: normalization layer. use_checkpoint: use gradient checkpointing for reduced memory usage. )r-r,r/rrr$swin) hidden_sizemlp_dimact dropout_rate dropout_modeN)rQrRrr,r-rr1r=norm1rrr rIdentityrnorm2r'Mlpmlp)rbrr,r-rr1r/rrrrr:r=Zmlp_hidden_dimrcrPrerRs(    zSwinTransformerBlock.__init__c CsP|}||}t|dkrq|j\}}}}}t|||f|j|j\} } d} } } | d|| d| d}| d|| d| d}| d|| d| d}t|dd| || || |f}|j\}}}}}||||g}nK|j\}}}}t||f|j|j\} } d} } | d|| d| d}| d|| d| d}t|dd| || |f}|j\}}}}|||g}t dd| Drt|dkrt j || d | d | d fdd}nt|d krt j || d | d fd d}|}n|}d}t || }|j ||d }|jd g| |fR}t|| |}t d d| DrTt|dkr>t j || d| d| dfdd}nt|d krSt j || d| dfd d}n|}t|dkr|dksl|dksl|dkr|ddd|d|d|ddf}|St|d kr|dks|dkr|ddd|d|ddf}|S)Nr|rrDrcs|]}|dkVqdSrNrP.0rrPrPre lz5SwinTransformerBlock.forward_part1..)rDrr)shiftsrrL)rDr)rrcsrrrPrrPrPreryr)rrrrrr-rFpadrrmrollrrrrr)rbr mask_matrixrrr{rrrr-rpad_lpad_tZpad_d0Zpad_d1pad_bpad_r_dphpwprZ shifted_x attn_maskZ x_windowsZ attn_windowsrPrPre forward_part1Ts^     *    $*$z"SwinTransformerBlock.forward_part1cCs||||SN)rrr)rbrrPrPre forward_part2sz"SwinTransformerBlock.forward_part2cCsd|d|d}gd}t|jj|d||d|jj|d||d|jj|d||d|jj|d||d |jj j|d||d |jj j|d||d |jj j|d||d |jj j|d||d |j j|d||d|j j|d||d|j j j|d||d|j j j|d||d|j jj|d||d|j jj|d||dWddS1swYdS)Nzmodule.z .0.blocks..)z norm1.weightz norm1.biasz!attn.relative_position_bias_tablezattn.relative_position_indexzattn.qkv.weightz attn.qkv.biaszattn.proj.weightzattn.proj.biasz norm2.weightz norm2.biaszmlp.fc1.weightz mlp.fc1.biaszmlp.fc2.weightz mlp.fc2.biasrfrrDrrrLr|rr!rN r )rmrnrrqrrrsrrrrrprrlinear1linear2)rbryrhrirootZ block_namesrPrPrervs$        ""zSwinTransformerBlock.load_fromcCsl|}|jrtj|j||dd}n|||}|||}|jr-|tj|j|dd}|S|||}|S)NF) use_reentrant)r= checkpointrrr)rbrrshortcutrPrPrers zSwinTransformerBlock.forward)rr'r,r'r-r+rr+r1r2r/r0rr2rr2rr2rrUr:r;r=r0rBrC) rrrrrrrRrrrvrrrPrPrcrers 56"rcs2eZdZdZejdfdfd d Zd d ZZS)rz Patch merging layer based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer rrr'r:r;r>rBrCcs|t||_|dkr"tjd|d|dd|_|d||_dS|dkrrcrPrerRs zPatchMergingV2.__init__c s:}t|dkrR|\}}}}}|ddkp"|ddkp"|ddk}|r9tddd|dd|dd|dftfddttdtdtdDdn?t|dkr|\}}}}|ddkpi|ddk}|r|tddd|dd|dftfd dttdtdDd| | S) Nr|rrDrc s>g|]\}}}dd|dd|dd|ddddfqSNrrP)rrjrrrPre s>z*PatchMergingV2.forward..rrLcs4g|]\}}dd|dd|ddddfqSr rP)rrr rrPrers4) rrrrrmcat itertoolsproductrrxrw) rbrrrr{rrr pad_inputrPrrers$ $((   *  zPatchMergingV2.forward)rr'r:r;r>r'rBrC rrrrrrrRrrrPrPrcrersrcs eZdZdZfddZZS)rz7The `PatchMerging` module previously defined in v0.9.0.c s0|}t|dkrt|St|dkrtd|jd|\}}}}}|ddkp7|ddkp7|ddk}|rNt|ddd|dd|dd|df}|dddddddddddddf} |dddddddddddddf} |dddddddddddddf} |dddddddddddddf} |dddddddddddddf} |dddddddddddddf}|dddddddddddddf}|dddddddddddddf}t | | | | | |||gd}| |}| |}|S) NrLr|zexpecting 5D x, got rrrDrr) rrrQrrSrrrrmrrxrw)rbrrrr{rrrrx0x1x2x3x4x5x6x7rcrPrers*   $(,,,,,,,,  zPatchMerging.forward)rrrrrrrPrPrcrersr)r%Z mergingv2c Csd}t|dkr|\}}}tjd|||df|d}t|d t|d |d t|d dfD]K}t|d t|d |d t|d dfD]/}t|d t|d |d t|d dfD]}||dd|||ddf<|d7}qhqMq2n]t|dkr|\}}tjd||df|d}t|d t|d |d t|d dfD].}t|d t|d |d t|d dfD]}||dd||ddf<|d7}qqt||} | d} | d| d} | | dktd| dktd } | S) adComputing region masks based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer Args: dims: dimension values. window_size: local window size. shift_size: shift size. device: device. rrrD)deviceNrrgYr$) rrmrslicersqueezer masked_fillr2) rr-rrcntr{rrZimg_maskZ mask_windowsrrPrPre compute_mask s2  666  66   $r"cs<eZdZdZddddejddfdfdd ZddZZS) rz Basic Swin Transformer layer in one stage based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer r"Fr$Nrr'depthr,r-r+rrr1r2r/r0rrr:r;r?nn.Module | Noner=rBrCc st|_tdd|D_tdd|D_|_ _t  f ddt |D_ | _ t j rO| tjd_ dSdS)a Args: dim: number of feature channels. depth: number of layers in each stage. num_heads: number of attention heads. window_size: local window size. drop_path: stochastic depth rate. mlp_ratio: ratio of mlp hidden dim to embedding dim. qkv_bias: add a learnable bias to query, key, value. drop: dropout rate. attn_drop: attention dropout rate. norm_layer: normalization layer. downsample: an optional downsampling layer at the end of the layer. use_checkpoint: use gradient checkpointing for reduced memory usage. css|]}|dVqdS)rNrPrrPrPrer\rz&BasicLayer.__init__..css|]}dVqdSrrPrrPrPrer]scsRg|]%}tj|ddkrjnjttr!|n d qS)rr) rr,r-rr1r/rrrr:r=)rr-no_shiftrrTrr rrrrr1r:r,r/rbr=rPreras z'BasicLayer.__init__..)rr:r>N)rQrRr-rrr%r#r=r ModuleListrrtr?callabler) rbrr#r,r-rr1r/rrr:r?r=rcr&rerR;s  zBasicLayer.__init__cCs|}t|dkr|\}}}}}t|||f|j|j\}} t|d}tt||d|d} tt||d|d} tt||d|d} t | | | g|| |j } |j D]}||| }q^| ||||d}|j dury| |}t|d}|St|dkr|\}}}}t||f|j|j\}} t|d }tt||d|d} tt||d|d} t | | g|| |j } |j D]}||| }q| |||d}|j dur| |}t|d }|S) Nr|zb c d h w -> b d h w crrDrrzb d h w c -> b c d h wrLzb c h w -> b h w czb h w c -> b c h w)rrrr-rrr'r}ceilr"rrtrr?)rbrrrrr{rrr-rrrrrblkrPrPrervs<              zBasicLayer.forward)rr'r#r'r,r'r-r+rrr1r2r/r0rr2rr2r:r;r?r$r=r0rBrCrrPrPrcrer3s;rc sPeZdZdZdddddejdddddf d$fdd Zd%d d!Zd&d"d#ZZ S)'rz Swin Transformer based on: "Liu et al., Swin Transformer: Hierarchical Vision Transformer using Shifted Windows " https://github.com/microsoft/Swin-Transformer r"Tr$Frr%rEr'rFr-r+r)r*r,r1r2r/r0r6r7rGr:r;r<r=r>rBrCcsPtt||_||_| |_||_||_t|j|||jr | nd|d|_ t j | d|_ ddt d| t|D}||_t |_t |_t |_t |_|jrit |_t |_t |_t |_t|trst|tn|}t|jD]}tt |d||||||j|t|d|t|d|d||| | | ||d }|dkr|j!|n |dkr|j!|n|dkr|j!|n |d kr|j!||jrt"||d||d|d dd d d }|dkr|j!|qz|dkr|j!|qz|dkr|j!|qz|d kr|j!|qzt |d|jd|_#dS)a Args: in_chans: dimension of input channels. embed_dim: number of linear projection output channels. window_size: local window size. patch_size: patch size. depths: number of layers in each stage. num_heads: number of attention heads. mlp_ratio: ratio of mlp hidden dim to embedding dim. qkv_bias: add a learnable bias to query, key, value. drop_rate: dropout rate. attn_drop_rate: attention dropout rate. drop_path_rate: stochastic depth rate. norm_layer: normalization layer. patch_norm: add normalization after patch embedding. use_checkpoint: use gradient checkpointing for reduced memory usage. spatial_dims: spatial dimension. downsample: module used for downsampling, available options are `"mergingv2"`, `"merging"` and a user-specified `nn.Module` following the API defined in :py:class:`monai.networks.nets.PatchMerging`. The default is currently `"merging"` (the original version defined in v0.9.0). use_v2: using swinunetr_v2, which adds a residual convolution block at the beginning of each swin stage. N)r)rErFr:r>)pcSsg|]}|qSrP)item)rrrPrPrersz,SwinTransformer.__init__..rrrD) rr#r,r-rr1r/rrr:r?r=rr#TrH)$rQrRr num_layersrFr<r-r)rrorrpos_droprmlinspacerrAr'rgrjrkrllayers1clayers2clayers3clayers4crTrUr rrrr'appendr num_features)rbrErFr-r)r*r,r1r/r6r7rGr:r<r=r>r?rAZdprZdown_sample_modi_layerriZlayercrcrPrerRs +          &        zSwinTransformer.__init__cCsz|r;|j}t|d}t|dkr$t|d}t||g}t|d}|St|dkr;t|d}t||g}t|d}|S)NrDr|zn c d h w -> n d h w czn d h w c -> n c d h wrLzn c h w -> n h w czn h w c -> n c h w)rr'rrr layer_norm)rbrr9rchrPrPreproj_outs      zSwinTransformer.proj_outc Cs||}||}|||}|jr|jd|}|jd|}|||}|jr7|jd|}|jd|}|||}|jrR|j d|}|j d|} || |} |jrm|j d| } |j d| } || |} |||| | gS)Nr) ror.r9rAr0rrgr1rjr2rkr3rl) rbrr9rZx0_outrZx1_outrZx2_outrZx3_outrZx4_outrPrPrers(       zSwinTransformer.forward) rEr'rFr'r-r+r)r+r*r+r,r+r1r2r/r0r6r2r7r2rGr2r:r;r<r0r=r0r>r'rBrC)F)T) rrrrrrrRr9rrrPrPrcrers  qrcCsl|dvrdS|dddkr4|dddkr"d|dd}||fSd|dd|d d}||fSdS) a A filter function used to filter the pretrained weights from [1], then the weights can be loaded into MONAI SwinUNETR Model. This function is typically used with `monai.networks.copy_model_state` [1] "Valanarasu JM et al., Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training " Args: key: the key in the source state dict used for the update. value: the value in the source state dict used for the update. Examples:: import torch from monai.apps import download_url from monai.networks.utils import copy_model_state from monai.networks.nets.swin_unetr import SwinUNETR, filter_swinunetr model = SwinUNETR(in_channels=1, out_channels=3, feature_size=48) resource = ( "https://github.com/Project-MONAI/MONAI-extra-test-data/releases/download/0.8.1/ssl_pretrained_weights.pth" ) ssl_weights_path = "./ssl_pretrained_weights.pth" download_url(resource, ssl_weights_path) ssl_weights = torch.load(ssl_weights_path, weights_only=True)["model"] dst_dict, loaded, not_loaded = copy_model_state(model, ssl_weights, filter_func=filter_swinunetr) )zencoder.mask_tokenzencoder.norm.weightzencoder.norm.biaszout.conv.conv.weightzout.conv.conv.biasNrNzencoder.rozswinViT.rP)keyvaluenew_keyrPrPrefilter_swinunetr6sr@r)0 __future__rrcollections.abcrnumpyr}rmtorch.nnrtorch.nn.functional functionalrtorch.utils.checkpointutilsr rmonai.networks.blocksrrrrrr monai.networks.layersr r monai.utilsr r rrr__all__Modulerrrrrrrrrr"rrr@rPrPrPres@     "" m'0 (d