U PhF@sddlmZddlmZmZmZddlmZddlZ ddl Z ddl m Z ddl mZddlmZddlmZmZmZmZmZddlmZGd d d ZdS) ) annotations)CallableMappingSequence)AnyN) MetaTensor) eval_mode)ComposeGaussianSmoothLambdaScaleIntensity SpatialCrop)ensure_tuple_repc @seZdZdZd-dddd d d d d dddZedd dddddZed.ddd ddddZedddddddd ddd dd Zed!ddd"d#d$d%Z d/dd'd(d)d*d+d,Z d&S)0OcclusionSensitivitya This class computes the occlusion sensitivity for a model's prediction of a given image. By occlusion sensitivity, we mean how the probability of a given prediction changes as the occluded section of an image changes. This can be useful to understand why a network is making certain decisions. As important parts of the image are occluded, the probability of classifying the image correctly will decrease. Hence, more negative values imply the corresponding occluded volume was more important in the decision process. Two ``torch.Tensor`` will be returned by the ``__call__`` method: an occlusion map and an image of the most probable class. Both images will be cropped if a bounding box used, but voxel sizes will always match the input. The occlusion map shows the inference probabilities when the corresponding part of the image is occluded. Hence, more -ve values imply that region was important in the decision process. The map will have shape ``BCHW(D)N``, where ``N`` is the number of classes to be inferred by the network. Hence, the occlusion for class ``i`` can be seen with ``map[...,i]``. The most probable class is an image of the probable class when the corresponding part of the image is occluded (equivalent to ``occ_map.argmax(dim=-1)``). See: R. R. Selvaraju et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. https://doi.org/10.1109/ICCV.2017.74. Examples: .. code-block:: python # densenet 2d from monai.networks.nets import DenseNet121 from monai.visualize import OcclusionSensitivity import torch model_2d = DenseNet121(spatial_dims=2, in_channels=1, out_channels=3) occ_sens = OcclusionSensitivity(nn_module=model_2d) occ_map, most_probable_class = occ_sens(x=torch.rand((1, 1, 48, 64)), b_box=[2, 40, 1, 62]) # densenet 3d from monai.networks.nets import DenseNet from monai.visualize import OcclusionSensitivity model_3d = DenseNet(spatial_dims=3, in_channels=1, out_channels=3, init_features=2, growth_rate=2, block_config=(6,)) occ_sens = OcclusionSensitivity(nn_module=model_3d, n_batch=10) occ_map, most_probable_class = occ_sens(torch.rand(1, 1, 6, 6, 6), b_box=[1, 3, -1, -1, -1, -1]) See Also: - :py:class:`monai.visualize.occlusion_sensitivity.OcclusionSensitivity.` Tgaussian?z nn.Modulezint | Sequenceintboolzstr | float | Callablefloatzbool | CallableNone) nn_module mask_sizen_batchverbosemodeoverlapactivatereturncCsD||_||_||_||_||_||_t|tr:|dkr:t||_ dS)at Occlusion sensitivity constructor. Args: nn_module: Classification model to use for inference mask_size: Size of box to be occluded, centred on the central voxel. If a single number is given, this is used for all dimensions. If a sequence is given, this is used for each dimension individually. n_batch: Number of images in a batch for inference. verbose: Use progress bar (if ``tqdm`` available). mode: what should the occluded region be replaced with? If a float is given, that value will be used throughout the occlusion. Else, ``gaussian``, ``mean_img`` and ``mean_patch`` can be supplied: * ``gaussian``: occluded region is multiplied by 1 - gaussian kernel. In this fashion, the occlusion will be 0 at the center and will be unchanged towards the edges, varying smoothly between. When gaussian is used, a weighted average will be used to combine overlapping regions. This will be done using the gaussian (not 1-gaussian) as occluded regions count more. * ``mean_patch``: occluded region will be replaced with the mean of occluded region. * ``mean_img``: occluded region will be replaced with the mean of the whole image. overlap: overlap between inferred regions. Should be in range 0<=x<1. activate: if ``True``, do softmax activation if num_channels > 1 else do ``sigmoid``. If ``False``, don't do any activation. If ``callable``, use callable on inferred outputs. )r mean_patchmean_imgN) rrrrrr isinstancestrNotImplementedErrorr)selfrrrrrrrr%Z/home/dell461/cl/sdc2/HISourceFinder-master-l/src/monai/visualize/occlusion_sensitivity.py__init__Ls#zOcclusionSensitivity.__init__z torch.Tensorrztuple[float, torch.Tensor])xvalrrcCs.tj|jdd||j|jd}d||fS)zVOcclude with a constant occlusion. Multiplicative is zero, additive is constant value.Ndevicedtyper)torchonesshaper,r-)r(r)rr/r%r%r&constant_occlusionzs"z'OcclusionSensitivity.constant_occlusionztuple[torch.Tensor, float])r(rsigmarcstj|jdf||j|jd}|jdd}tdgdd|D}d||<ttfdd|Ddtd d t g}||d}|d fS) z For Gaussian occlusion, Multiplicative is 1-Gaussian, additive is zero. Default sigma of 0.25 empirically shown to give reasonable kernel, see here: https://github.com/Project-MONAI/MONAI/pull/5230#discussion_r984520714. r+NcSs"g|]}t|d|ddqS)r*r3slice).0sr%r%r& sz;OcclusionSensitivity.gaussian_occlusion..g?csg|] }|qSr%r%r6br2r%r&r8sr;cSs| SNr%)r(r%r%r&z9OcclusionSensitivity.gaussian_occlusion..r) r.zerosr0r,r-r5r r r r )r(rr2kernel spatial_shapecenterrmulr%r;r&gaussian_occlusions & z'OcclusionSensitivity.gaussian_occlusionztorch.Tensor | floatr"zMapping[str, Any]) cropped_gridrr(rCaddrocc_moder module_kwargsrc CsZ|jd} |jd} t|| d} tdgdtdg| } || } t| D]\}}t||dtdgddt||D}| |}|dkrt|| |\}}t |r|||}n |||}|dks|dkrt d|| |<qN|| f|}t |r ||}n(|r4|jddkr*| n|d}|D]}tj|d |d d }q8|S) aT Predictor function to be passed to the sliding window inferer. Takes a cropped meshgrid, referring to the coordinates in the input image. We use the index of the top-left corner in combination ``mask_size`` to figure out which region of the image is to be occluded. The occlusion is performed on the original image, ``x``, using ``cropped_region * mul + add``. ``mul`` and ``add`` are sometimes pre-computed (e.g., a constant Gaussian blur), or they are sometimes calculated on the fly (e.g., the mean of the occluded patch). For this reason ``occ_mode`` is given. Lastly, ``activate`` is used to activate after each call of the model. Args: cropped_grid: subsection of the meshgrid, where each voxel refers to the coordinate of the input image. The meshgrid is created by the ``OcclusionSensitivity`` class, and the generation of the subset is determined by ``sliding_window_inference``. nn_module: module to call on data. x: the image that was originally passed into ``OcclusionSensitivity.__call__``. mul: occluded region will be multiplied by this. Can be ``torch.Tensor`` or ``float``. add: after multiplication, this is added to the occluded region. Can be ``torch.Tensor`` or ``float``. mask_size: Size of box to be occluded, centred on the central voxel. Should be a sequence, one value for each spatial dimension. occ_mode: might be used to calculate ``mul`` and ``add`` on the fly. activate: if ``True``, do softmax activation if num_channels > 1 else do ``sigmoid``. If ``False``, don't do any activation. If ``callable``, use callable on inferred outputs. module_kwargs: kwargs to be passed onto module when inferring rr*Nr3cSs&g|]\}}tt|t||qSr%)r5r)r6jmr%r%r&r8sz2OcclusionSensitivity.predictor..rz,Shouldn't be here, something's gone wrong...)dim)r0ndimr.repeat_interleaver5 enumerateziprr1meanitemcallable RuntimeErrorsigmoidsoftmax unsqueeze)rErr(rCrFrrGrrHrsdimZcorner_coord_slicesZ top_cornersr:tslicesZ to_occludeoutrJr%r%r& predictors0$  *      "zOcclusionSensitivity.predictorrz(tuple[MetaTensor, SpatialCrop, Sequence])gridb_boxrrc Csdd|D}ddt|ddd|D}g}t|ddd||jddD]2\}}}|dkrn||qP|t|||qPddt||D} t| d } | |d d} t|}t| jddD]\} }t||| || <q| | |fS) zXCrop the meshgrid so we only perform occlusion sensitivity on a subsection of the image.cSsg|]}|ddqS)r3r*r%)r6rJr%r%r&r8sz6OcclusionSensitivity.crop_meshgrid..cSsg|]\}}t||dqSrmax)r6r:rJr%r%r&r8sNr*r3rKcSsg|]\}}t||qSr%r4)r6r7er%r%r&r8s) roi_slicesr)rPr0appendminr listrO) r^r_rZ mask_edgebbox_minbbox_maxr:rJr7r[croppercroppedir%r%r& crop_meshgrids*  z"OcclusionSensitivity.crop_meshgridNzSequence | Nonerz!tuple[torch.Tensor, torch.Tensor])r(r_kwargsrcKs*|jddkrtd|jd}t|j|}tttjdd|jddDdd id|j |j d }|dk r| |||\}}}t d d t |jdd|Drtd |jddd|dt|jtr|||j|\}} nN|jdkr||||\}} n&|jdkr.|||\}} nd\} }t|jVddlm} | |||jtj|j|jdkrpdnd|j|j|| |||j|j|d} W5QRX|dk r|| dd} dd|dddD} ddt |ddd|jddD} t | | d}|| dd} | j!ddd}| |fS)a Args: x: Image to use for inference. Should be a tensor consisting of 1 batch. b_box: Bounding box on which to perform the analysis. The output image will be limited to this size. There should be a minimum and maximum for all spatial dimensions: ``[min1, max1, min2, max2,...]``. * By default, the whole image will be used. Decreasing the size will speed the analysis up, which might be useful for larger images. * Min and max are inclusive, so ``[0, 63, ...]`` will have size ``(64, ...)``. * Use -ve to use ``min=0`` and ``max=im.shape[x]-1`` for xth dimension. * N.B.: we add half of the mask size to the bounding box to ensure that the region of interest has a sufficiently large area surrounding it. kwargs: any extra arguments to be passed on to the module as part of its `__call__`. Returns: * Occlusion map: * Shows the inference probabilities when the corresponding part of the image is occluded. Hence, more -ve values imply that region was important in the decision process. * The map will have shape ``BCHW(D)N``, where N is the number of classes to be inferred by the network. Hence, the occlusion for class ``i`` can be seen with ``map[...,i]``. * If `per_channel==False`, output ``C`` will equal 1: ``B1HW(D)N`` * Most probable class: * The most probable class when the corresponding part of the image is occluded (``argmax(dim=-1)``). Both images will be cropped if a bounding box used, but voxel sizes will always match the input. rr3zExpected batch size of 1.r*cSsg|]}td|qSr`)nparange)r6rlr%r%r&r8!sz1OcclusionSensitivity.__call__..Nindexingijr+css|]\}}||kVqdSr<r%)r6grJr%r%r& *sz0OcclusionSensitivity.__call__..zImage (spatial shape) z should be bigger than mask .r r)NN)sliding_window_inferenceconstant)roi_size sw_batch_sizer]rrprogressrr(rFrCrrGrrHcSsg|]}t|dqSr`rar9r%r%r&r8WscSs g|]\}}|dkr|n|qSr`r%)r6r:r7r%r%r&r8Xs) roi_startroi_endT)rLkeepdim)"r0 ValueErrorrMrrrrostackmeshgridr,r-rmanyrPr!rrr1rQrRrDrrmonai.inferersrvrrr]rrrinverser argmax)r$r(r_rnrXrr^rjrCrFrvZsensitivity_imrhriZmost_probable_classr%r%r&__call__s^  ,"       ( zOcclusionSensitivity.__call__)rrTrrT)r)N) __name__ __module__ __qualname____doc__r' staticmethodr1rDr]rmrr%r%r%r&rs$3."Kr) __future__rcollections.abcrrrtypingrnumpyror.torch.nnnnmonai.data.meta_tensorrmonai.networks.utilsrmonai.transformsr r r r r monai.utilsrrr%r%r%r& s