o  i>@sddlmZddlZddlmZmZddlmZddlZ ddl Z ddl m Z ddl m mZddlmZddlmZddlmZddlmZgd Zdd dZGdddZGdddZGdddeZGdddeZGdddeZdS)) annotationsN)CallableSequence)cast) NdarrayTensor)ScaleIntensity) ensure_tuple)default_upsampler)CAMGradCAM GradCAMppModelWithHooksdefault_normalizerxrreturncCs>ddd}t|tjrtj|||jdS||S) a/ A linear intensity scaling by mapping the (min, max) to (1, 0). If the input data is PyTorch Tensor, the output data will be Tensor on the same device, otherwise, output data will be numpy array. Note: This will flip magnitudes (i.e., smallest will become biggest and vice versa). data np.ndarrayrcs(tdddtjfdd|DddS)Ng?)minvmaxvcsg|]}|qSr).0iscalerrg/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/visualize/class_activation_maps.py *sz8default_normalizer.._compute..r)axis)rnpstack)rrrr_compute(s z$default_normalizer.._compute)deviceN)rrrr) isinstancetorchTensor as_tensordetachcpunumpyr!)rr rrrrs  rc@sVeZdZdZ  d d!d d Zd d ZddZd"ddZd#ddZd$ddZ ddZ dS)%r zy A model wrapper to run model forward/backward steps and storing some intermediate feature/gradient information. F nn_module nn.Moduletarget_layer_namesstr | Sequence[str]register_forwardboolregister_backwardcCs||_t||_i|_i|_d|_d|_||_||_g}| D]4\}}||jvr*q | ||jrId|j vrA|j drAd|j d<| | ||jrT|||q |jrmt|t|jkrotd|jddSdSdS)ag Args: nn_module: the model to be wrapped. target_layer_names: the names of the layer to cache. register_forward: whether to cache the forward pass output corresponding to `target_layer_names`. register_backward: whether to cache the backward pass output corresponding to `target_layer_names`. NinplaceFz._hookrrBrDrLrrKrr<_zModelWithHooks.backward_hookcrG)Ncs|j<dSNr5)rI_inputoutputrKrrrLhsz*ModelWithHooks.forward_hook.._hookrrMrrKrr>frNzModelWithHooks.forward_hooklayer_id&str | Callable[[nn.Module], nn.Module]rcCsXt|r ||jSt|tr$|jD]\}}||kr#ttj|Sqtd|d)z Args: layer_id: a layer name string or a callable. If it is a callable such as `lambda m: m.fc`, this method will return the module `self.model.fc`. Returns: a submodule from self.model. zCould not find r1) callabler2r"strr8rnnModuleNotImplementedError)rBrSrDrErrr get_layerms  zModelWithHooks.get_layerlogits torch.Tensorr7intcCs|dd|fSrO)squeeze)rBr[r7rrr class_scoreszModelWithHooks.class_scoreNc  sjj}jj|fi|}|dur|ddn|_d\}}jr3tfddjD}jrr |t t j_ j j j|djD]} | jvretd| d| d qRtfd djD}|ryj|||fS) N)NNc3s|]}j|VqdSrOrPrlayerrBrr sz*ModelWithHooks.__call__..) retain_graphzBackward hook for z& is not triggered; `requires_grad` of z should be `True`.c3s$|] }|jvrj|VqdSrOrHrbrdrrres")r2trainingevalmaxr7r-tupler3r/r_rr]r6 zero_gradsumbackwardr4r@rAtrain) rBrr7rfkwargsrnr[actigradrcrrdr__call__s*      zModelWithHooks.__call__cCs|jSrO)r2rdrrrget_wrapped_netzModelWithHooks.get_wrapped_net)FF)r)r*r+r,r-r.r/r.)rSrTrr*)r[r\r7r]rr\)NF) __name__ __module__ __qualname____doc__rFr<r>rZr_rrrsrrrrr 2s (    r c@sFeZdZdZeedfdddZdddZdddZddZ ddZ dS)CAMBasez% Base class for CAM methods. Tr)r*r3rV upsamplerrpostprocessingr/r.rNonecCs8|t|tst||d|d|_n||_||_||_dS)NT)r-r/)r"r r)rzr{)rBr)r3rzr{r/rrrrFs   zCAMBase.__init__r'racKs$|jtj|d|ifd|i|jS)a Computes the actual feature map size given `nn_module` and the target_layer name. Args: input_size: shape of the input tensor device: the device used to initialise the input tensor layer_idx: index of the target layer if there are multiple target layers. Defaults to -1. kwargs: any extra arguments to be passed on to the module as part of its `__call__`. Returns: shape of the actual feature map. r! layer_idx) compute_mapr#zerosshape)rB input_sizer!r}rorrrfeature_map_sizes$ zCAMBase.feature_map_sizeNcCt)a Compute the actual feature map with input tensor `x`. Args: x: input to `nn_module`. class_idx: index of the class to be visualized. Default to `None` (computing `class_idx` from `argmax`) layer_idx: index of the target layer if there are multiple target layers. Defaults to -1. Returns: activation maps (raw outputs without upsampling/post-processing.) rY)rBrr7r}rrrr~s zCAMBase.compute_mapcCs&|jdd}|||}||S)N)rrzr{)rBacti_maprZ img_spatialrrr_upsample_and_post_processs z"CAMBase._upsample_and_post_processcCrrOrrdrrrrrrtzCAMBase.__call__) r)r*r3rVrzrr{rr/r.rr|)r'raNra) rurvrwrxr rrFrr~rrrrrrrrys   rycs>eZdZdZdeefdfdd ZdddZdddZZ S)r a Compute class activation map from the last fully-connected layers before the spatial pooling. This implementation is based on: Zhou et al., Learning Deep Features for Discriminative Localization. CVPR '16, https://arxiv.org/abs/1512.04150 Examples .. code-block:: python import torch # densenet 2d from monai.networks.nets import DenseNet121 from monai.visualize import CAM model_2d = DenseNet121(spatial_dims=2, in_channels=1, out_channels=3) cam = CAM(nn_module=model_2d, target_layers="class_layers.relu", fc_layers="class_layers.out") result = cam(x=torch.rand((1, 1, 48, 64))) # resnet 2d from monai.networks.nets import seresnet50 from monai.visualize import CAM model_2d = seresnet50(spatial_dims=2, in_channels=3, num_classes=4) cam = CAM(nn_module=model_2d, target_layers="layer4", fc_layers="last_linear") result = cam(x=torch.rand((2, 3, 48, 64))) N.B.: To help select the target layer, it may be useful to list all layers: .. code-block:: python for name, _ in model.named_modules(): print(name) See Also: - :py:class:`monai.visualize.class_activation_maps.GradCAM` fcr)r*r3rV fc_layersstr | Callablerzrr{rr|cs tj||||dd||_dS)a' Args: nn_module: the model to be visualized target_layers: name of the model layer to generate the feature map. fc_layers: a string or a callable used to get fully-connected weights to compute activation map from the target_layers (without pooling). and evaluate it at every spatial location. upsampler: An upsampling method to upsample the output image. Default is N dimensional linear (bilinear, trilinear, etc.) depending on num spatial dimensions of input. postprocessing: a callable that applies on the upsampled output image. Default is normalizing between min=1 and max=0 (i.e., largest input will become 0 and smallest input will become 1). F)r)r3rzr{r/N)superrFr)rBr)r3rrzr{ __class__rrrFs z CAM.__init__Nrac  s|j|fi|\}}}||}|dur|dd}|j^}} } tj||| dddd}|j|jtjfdd|Dddtjfddt |Dddj|dg| RS) Nr`rar)dimcsg|]}|dqS)).rr)ra)rrrr*sz#CAM.compute_map..cs$g|]\}}|||dfqS)r`r)rrb)rRrrr+s$r) r)rirr#splitreshaperZrr enumerate) rBrr7r}ror[rp_rcspatialr)rrRrr~"s  zCAM.compute_mapcKs"|j|||fi|}|||S)a Compute the activation map with upsampling and postprocessing. Args: x: input tensor, shape must be compatible with `nn_module`. class_idx: index of the class to be visualized. Default to argmax(logits) layer_idx: index of the target layer if there are multiple target layers. Defaults to -1. kwargs: any extra arguments to be passed on to the module as part of its `__call__`. Returns: activation maps r~r)rBrr7r}rorrrrrr.s z CAM.__call__) r)r*r3rVrrrzrr{rrr|r) rurvrwrxr rrFr~rr __classcell__rrrrr s-  r c@s$eZdZdZd ddZd ddZdS) r a Computes Gradient-weighted Class Activation Mapping (Grad-CAM). This implementation is based on: Selvaraju et al., Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization, https://arxiv.org/abs/1610.02391 Examples .. code-block:: python import torch # densenet 2d from monai.networks.nets import DenseNet121 from monai.visualize import GradCAM model_2d = DenseNet121(spatial_dims=2, in_channels=1, out_channels=3) cam = GradCAM(nn_module=model_2d, target_layers="class_layers.relu") result = cam(x=torch.rand((1, 1, 48, 64))) # resnet 2d from monai.networks.nets import seresnet50 from monai.visualize import GradCAM model_2d = seresnet50(spatial_dims=2, in_channels=3, num_classes=4) cam = GradCAM(nn_module=model_2d, target_layers="layer4") result = cam(x=torch.rand((2, 3, 48, 64))) N.B.: To help select the target layer, it may be useful to list all layers: .. code-block:: python for name, _ in model.named_modules(): print(name) See Also: - :py:class:`monai.visualize.class_activation_maps.CAM` NFracKs|j|f||d|\}}}||||}}|j^} } } || | ddj| | gdgt| R} | |jddd} t| S)Nr7rfrarr`Tkeepdim)r)rviewmeanr?rlFrelu)rBrr7rfr}rorrprqrrrweightsrrrrr~is  . zGradCAM.compute_mapcKs&|j|f|||d|}|||S)aD Compute the activation map with upsampling and postprocessing. Args: x: input tensor, shape must be compatible with `nn_module`. class_idx: index of the class to be visualized. Default to argmax(logits) layer_idx: index of the target layer if there are multiple target layers. Defaults to -1. retain_graph: whether to retain_graph for torch module backward call. kwargs: any extra arguments to be passed on to the module as part of its `__call__`. Returns: activation maps )r7rfr}r)rBrr7r}rfrorrrrrrqs zGradCAM.__call__NFra)NraF)rurvrwrxr~rrrrrrr ?s )r c@seZdZdZdddZdS)r aW Computes Gradient-weighted Class Activation Mapping (Grad-CAM++). This implementation is based on: Chattopadhyay et al., Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks, https://arxiv.org/abs/1710.11063 See Also: - :py:class:`monai.visualize.class_activation_maps.GradCAM` NFracKs|j|f||d|\}}}||||}}|j^} } } |d} | d||d| | ddj| | gdgt| R} t| dk| t | } | | d}t t tj|jj|}||| | ddj| | gdgt| R}||jddd }t |S) Nrrrar`rgHz>Tr)r)rpowmulrrlr?r#where ones_likedivrrrr$r6exp)rBrr7rfr}rorrprqrrrZalpha_nrZalpha_dralphaZ relu_gradrrrrrr~s  D2 zGradCAMpp.compute_mapr)rurvrwrxr~rrrrr s r )rrrr) __future__rr@collections.abcrrtypingrr(rr#torch.nnrWtorch.nn.functional functionalr monai.configrmonai.transformsr monai.utilsrZmonai.visualize.visualizerr __all__rr ryr r r rrrrs&        j>eD