U Ph @sddlmZddlZddlmZmZddlZddlZddl m Z m Z ddl m Z mZmZmZmZmZmZddlmZddlmZmZmZmZmZddlmZmZmZdd l m!Z!dd l"m#Z#m$Z$ed d d \Z%Z&eddd \Z'Z&eddd \Z(Z&eddd \Z)Z&eddd \Z*Z&eddd \Z+Z&ddddddddddd g Z,Gd!ddeZ-Gd"ddeZ.Gd#ddeZ/Gd$ddeZ0Gd%ddeZ1Gd&ddeZ2Gd'ddeZ3Gd(ddeZ4Gd)ddeZ5Gd*ddeZ6Gd+d d eZ7dS),) annotationsN)CallableSequence) DtypeLikeNdarrayOrTensor) Activations AsDiscrete BoundingRect FillHolesGaussianSmoothRemoveSmallObjectsSobelGradients) Transform)maxmaximumminsumunique)TransformBackendsconvert_to_numpyoptional_import)ensure_tuple_rep)convert_to_dst_typeconvert_to_tensorzscipy.ndimage.measurementslabel)namezskimage.morphologydiskopeningzskimage.segmentation watershedzskimage.measure find_contourscentroid WatershedGenerateWatershedMaskGenerateInstanceBorderGenerateDistanceMapGenerateWatershedMarkersGenerateSuccinctContourGenerateInstanceContourGenerateInstanceCentroidGenerateInstanceType!HoVerNetInstanceMapPostProcessing!HoVerNetNuclearTypePostProcessingc@sHeZdZdZejgZdejfddddddZ dd d d d d d dZ d S)r!a Use `skimage.segmentation.watershed` to get instance segmentation results from images. See: https://scikit-image.org/docs/stable/api/skimage.segmentation.html#skimage.segmentation.watershed. Args: connectivity: an array with the same number of dimensions as image whose non-zero elements indicate neighbors for connection. Following the scipy convention, default is a one-connected array of the dimension of the image. dtype: target data content type to convert, default is np.int64.  int | NonerNone) connectivitydtypereturncCs||_||_dSN)r/r0)selfr/r0r4_/home/dell461/cl/sdc2/HISourceFinder-master-l/src/monai/apps/pathology/transforms/post/array.py__init__IszWatershed.__init__NrzNdarrayOrTensor | None)imagemaskmarkersr1cCs>t|}t|}t|}t||||jd}t|||jddS)a\ Args: image: image where the lowest value points are labeled first. Shape must be [1, H, W, [D]]. mask: optional, the same shape as image. Only points at which mask == True will be labeled. If None (no mask given), it is a volume of all 1s. markers: optional, the same shape as image. The desired number of markers, or an array marking the basins with the values to be assigned in the label matrix. Zero means not a marker. If None (no markers given), the local minima of the image are used as markers. )r9r8r/r0r)rrr/rr0)r3r7r8r9Z instance_segr4r4r5__call__Ms zWatershed.__call__)NN __name__ __module__ __qualname____doc__rNUMPYbackendnpint64r6r;r4r4r4r5r!:s  c@sJeZdZdZejgZdddejfddddd d d d Z d d dddZ dS)r"az generate mask used in `watershed`. Only points at which mask == True will be labeled. Args: activation: the activation layer to be applied on the input probability map. It can be "softmax" or "sigmoid" string, or any callable. Defaults to "softmax". threshold: an optional float value to threshold to binarize probability map. If not provided, defaults to 0.5 when activation is not "softmax", otherwise None. min_object_size: objects smaller than this size (in pixel) are removed. Defaults to 10. dtype: target data content type to convert, default is np.uint8. softmaxN str | Callable float | Noneintrr.) activation thresholdmin_object_sizer0r1cCs||_d}d}d}t|trR|dkr.d}qt|dkr@d}qttd|dn"t|r`|}ntdt|dt|||d|_|s|dkrd }t ||d |_ |d krt |d nd|_ dS) NFrETsigmoidJThe activation should be 'softmax' or 'sigmoid' string, or any callable. ' ' was given.7The activation type should be either str or callable. 'rErMother?rKargmaxrmin_size) r0 isinstancestrlower ValueErrorcallabletyperrJr as_discreter remove_small_objects)r3rJrKrLr0 use_softmax use_sigmoid activation_fnr4r4r5r6ss(     zGenerateWatershedMask.__init__r)prob_mapr1cCs\||}||}t|}t|d}|jdk r<||}d||dk<t|||jddS)zk Args: prob_map: probability map of segmentation, shape must be [C, H, W, [D]] rNr,r:)rJr^rrr_rr0)r3rcpredr4r4r5r;s      zGenerateWatershedMask.__call__) r=r>r?r@rrArBrCuint8r6r;r4r4r4r5r"cs $c@sDeZdZdZejgZdejfddddddZ d d d d d d Z d S)r#a Generate instance border by hover map. The more parts of the image that cannot be identified as foreground areas, the larger the grey scale value. The grey value of the instance's border will be larger. Args: kernel_size: the size of the Sobel kernel. Defaults to 5. dtype: target data type to convert to. Defaults to np.float32. Raises: ValueError: when the `mask` shape is not [1, H, W]. ValueError: when the `hover_map` shape is not [2, H, W]. rIrr.) kernel_sizer0r1cCs||_t|d|_dS)Nrg)r0r sobel_gradient)r3rgr0r4r4r5r6szGenerateInstanceBorder.__init__r)r8 hover_mapr1cCst|jdkr td|jdt|jdkrT|jddkr~td|jddn*t|jdkrl|d}ntd |jd|jddkrtd |jd|ddd f}|ddd f}t|t|}}t|t|}}||dks||dkrtd ||||}||||}||d } ||d} t| t| } } t| t| } }| | dks|| dkrtdd| | | | } d| | || } t| | }|d|}d||dk<t|||jddS)a Args: mask: binary segmentation map, the output of :py:class:`GenerateWatershedMask`. Shape must be [1, H, W] or [H, W]. hover_map: horizontal and vertical distances of nuclear pixels to their centres of mass. Shape must be [2, H, W]. The first and second channel represent the horizontal and vertical maps respectively. For more details refer to papers: https://arxiv.org/abs/1812.06499. z:The hover map should have the shape of [C, H, W], but got .rr,/The mask should have only one channel, but got N?The mask should have the shape of [1, H, W] or [H, W], but got z5Suppose the hover map only has two channels, but got .z.Not a valid hover map, please check your input)r,.)r.zNot a valid sobel gradient mapr:) lenshaper[rrrirrr0)r3r8rjZhover_hZhover_vZ hover_h_minZ hover_h_maxZ hover_v_minZ hover_v_maxZsobelhZsobelvZ sobelh_minZ sobelh_maxZ sobelv_minZ sobelv_maxZoverallr4r4r5r;s<     zGenerateInstanceBorder.__call__N r=r>r?r@rrArBrCfloat32r6r;r4r4r4r5r#sc@sDeZdZdZejgZdejfddddddZ d d d d d d Z dS) r$a Generate distance map. In general, the instance map is calculated from the distance to the background. Here, we use 1 - "instance border map" to generate the distance map. Nuclei values form mountains so invert them to get basins. Args: smooth_fn: smoothing function for distance map, which can be any callable object. If not provided :py:class:`monai.transforms.GaussianSmooth()` is used. dtype: target data type to convert to. Defaults to np.float32. NCallable | Nonerr.) smooth_fnr0r1cCs|dk r |nt|_||_dSr2)r rur0)r3rur0r4r4r5r6szGenerateDistanceMap.__init__rr8instance_borderr1cCst|jdkr4|jddkr^td|jddn*t|jdkrL|d}ntd|jd|jddksv|jdkrtd |jd ||}||}t| ||jd dS) 3 Args: mask: binary segmentation map, the output of :py:class:`GenerateWatershedMask`. Shape must be [1, H, W] or [H, W]. instance_border: instance border map, the output of :py:class:`GenerateInstanceBorder`. Shape must be [1, H, W]. rkrr,rmrlrnNro@Input instance_border should be with size of [1, H, W], but got ?r:)rprqr[ndimrurr0)r3r8rw distance_mapr4r4r5r;s   zGenerateDistanceMap.__call__rrr4r4r4r5r$s c@sPeZdZdZejgZddddejfddddd d d d d Z ddddddZ dS)r%a" Generate markers to be used in `watershed`. The watershed algorithm treats pixels values as a local topography (elevation). The algorithm floods basins from the markers until basins attributed to different markers meet on watershed lines. Generally, markers are chosen as local minima of the image, from which basins are flooded. Here is the implementation from HoVerNet paper. For more details refer to papers: https://arxiv.org/abs/1812.06499. Args: threshold: a float value to threshold to binarize instance border map. It turns uncertain area to 1 and other area to 0. Defaults to 0.4. radius: the radius of the disk-shaped footprint used in `opening`. Defaults to 2. min_object_size: objects smaller than this size (in pixel) are removed. Defaults to 10. postprocess_fn: additional post-process function on the markers. If not provided, :py:class:`monai.transforms.post.FillHoles()` will be used. dtype: target data type to convert to. Defaults to np.int64. 皙?rnrFNfloatrIrtrr.)rKradiusrLpostprocess_fnr0r1cCsB||_||_||_|dkr t}||_|dkr8t|dnd|_dS)NrrV)rKrr0r rr r_)r3rKrrLrr0r4r4r5r6/sz!GenerateWatershedMarkers.__init__rrvcCs t|jdkr4|jddkr^td|jddn*t|jdkrL|d}ntd|jd|jddksv|jdkrtd |j||jk}|t||d}d||dk<||}t|}t| t |j }t |dd}|j dk r| |}t|||jd dS) rxrkrr,rmrlrnNroryr:)rprqr[r{rKrrrrsqueezerrrr_r0)r3r8rwmarkerr4r4r5r;@s$      z!GenerateWatershedMarkers.__call__r<r4r4r4r5r%sc@sTeZdZdZddddddZdddd d d Zd dd ddZdddddZdS)r&aW Converts SciPy-style contours (generated by skimage.measure.find_contours) to a more succinct version which only includes the pixels to which lines need to be drawn (i.e. not the intervening pixels along each line). Args: height: height of bounding box, used to detect direction of line segment. width: width of bounding box, used to detect direction of line segment. Returns: the pixels that need to be joined by straight lines to describe the outmost pixels of the foreground similar to OpenCV's cv.CHAIN_APPROX_SIMPLE (counterclockwise) rIr.)heightwidthr1cCs||_||_dSr2)rr)r3rrr4r4r5r6osz GenerateSuccinctContour.__init__ np.ndarrayztuple[int, int])currentpreviousr1cCs|d|d|d|df}|dkrFt|dd}t|d}np|dkrht|d}t|d}nN|dkrt|d}t|dd}n(|dkrt|dd}t|dd}||fS)aR Generate contour coordinates. Given the previous and current coordinates of border positions, returns the int pixel that marks the extremity of the segmented pixels. Args: current: coordinates of the current border position. previous: coordinates of the previous border position. rr,))rz)rSrS)rzrrS))rg)rS))r)rr)rrS)rI)r3rrZp_deltarowcolr4r4r5_generate_contour_coordss   z/GenerateSuccinctContour._generate_contour_coordzSequence[tuple[int, int]])sequencer1cCs|d}|ddkr|d}nf|d|jdkr@|j|d}nD|d|jdkrld|j|j|d}nd|j|j|d}|S)a Each sequence of coordinates describes a boundary between foreground and background starting and ending at two sides of the bounding box. To order the sequences correctly, we compute the distance from the top-left of the bounding box around the perimeter in a clockwise direction. Args: sequence: list of border points coordinates. Returns: the distance round the perimeter of the bounding box from the top-left origin rr,rn)rr)r3rZ first_coorddistancer4r4r5!_calculate_distance_from_top_lefts   z9GenerateSuccinctContour._calculate_distance_from_top_leftzlist[np.ndarray])contoursr1cCsg}g}ddddg}|D]}g}d}d}d} t|D]d\} } | dkrx| ddkrd} dt| ddf} | d|jdkrd|d<n| ddkrd|d<n| ddkrd} t| dddf} n| d|jdkrd } t| dt| ddf} | d|jdkrhd|d <nP| d|jdkrNd } t| ddt| df} ntd | d dS|| | }n|| t|dkr|| |} | |kr|| | }nDt | ||| d| kr|| |} | |kr|| | }| t|dkr| dkr(| ddkrd|| <np| dkrP| d|jdkrd|| <nH| d krx| d|jdkrd|| <n | d kr| ddkrd|| <| }q6| |} || |d q|ddkr|ddgd |ddkr||jd|jdfgd |d dkr>||j|j|jd|jdfgd |d dkrt|d |j|j|jddfgd |j dddd}|D]D}|dd|kr| |r||d}n|d}|d}q|d|kr| d|ddkr|| dt t|t jdS)a Args: contours: list of (n, 2)-ndarrays, scipy-style clockwise line segments, with lines separating foreground/background. Each contour is an ndarray of shape (n, 2), consisting of n (row, column) coordinates along the contour. FNrrrr,rSTrkrnzInvalid contour coord z" is generated, skip this instance.)rrrrcSs |dS)Nr)getxr4r4r5z2GenerateSuccinctContour.__call__..)key)rrrr:) enumeraterIrrwarningswarnappendrprrCanyrsortpopfliprint32)r3rpixels sequencescornersgrouprZ last_addedprevcornericoordpixeldistlastZ _sequencer4r4r5r;s                         *(  z GenerateSuccinctContour.__call__N)r=r>r?r@r6rrr;r4r4r4r5r&as  c@s@eZdZdZejgZddddddd Zdd d d dddZdS)r'a  Generate contour for each instance in a 2D array. Use `GenerateSuccinctContour` to only include the pixels to which lines need to be drawn Args: min_num_points: assumed that the created contour does not form a contour if it does not contain more points than the specified value. Defaults to 3. contour_level: an optional value for `skimage.measure.find_contours` to find contours in the array. If not provided, the level is set to `(max(image) + min(image)) / 2`. rkNrIrHr.)min_num_points contour_levelr1cCs||_||_dSr2)rr)r3rrr4r4r5r6$sz GenerateInstanceContour.__init__rrzSequence[int] | Noneznp.ndarray | None inst_maskoffsetr1cCs|}t|}t||jd}t|jd|jd}||}|dkrHdS|jd|jkrntd|jddSt|jdkrtt|jddS|dddf|d7<|dddf|d7<|SdS) z Args: inst_mask: segmentation mask for a single instance. Shape should be [1, H, W, [D]] offset: optional offset of starting position of the instance mask in the original array. Default to 0 for each dim. )levelrr,Nz< z) points don't make a contour, so skipped!rnz != 2, check for tricky shapes!) rrrrr&rqrprintrp)r3rrZinst_contour_cvZgenerate_contourZ inst_contourr4r4r5r;(s z GenerateInstanceContour.__call__)rkN)r) r=r>r?r@rrArBr6r;r4r4r4r5r's c@s@eZdZdZejgZefdddddZddd dd d d Z d S)r(z Generate instance centroid using `skimage.measure.centroid`. Args: dtype: the data type of output centroid. zDtypeLike | Noner.)r0r1cCs ||_dSr2r:)r3r0r4r4r5r6Osz!GenerateInstanceCentroid.__init__rrzSequence[int] | intrcCsdt|}|d}t|j}t||}t|}t|D]}||||7<q6t|||jddS)z Args: inst_mask: segmentation mask for a single instance. Shape should be [1, H, W, [D]] offset: optional offset of starting position of the instance mask in the original array. Default to 0 for each dim. rr:) rrrprqrr rangerr0)r3rrr{Z inst_centroidrr4r4r5r;Rs    z!GenerateInstanceCentroid.__call__N)r) r=r>r?r@rrArBrIr6r;r4r4r4r5r(Dsc@s.eZdZdZejgZddddddddZd S) r)zC Generate instance type and probability for each instance. rrrIztuple[int, float]) type_predseg_predbbox instance_idr1cCs|\}}}}|d||||f} |d||||f} t| |k| tdd} | | } t| dd\} } tt| | } t| dddd} | dd} | dkrt| dkr| dd} d d | D}|| t| d }t | t |fS) ak Args: type_pred: pixel-level type prediction map after activation function. seg_pred: pixel-level segmentation prediction map after activation function. bbox: bounding box coordinates of the instance, shape is [channel, 2 * spatial dims]. instance_id: get instance type from specified instance id. rr:T) return_countscSs|dS)Nr,r4rr4r4r5rrz/GenerateInstanceType.__call__..)rreverser,cSsi|]}|d|dqS)rr,r4).0vr4r4r5 sz1GenerateInstanceType.__call__..gư>) flattenrboolrlistzipsortedrprrIr~)r3rrrrrminrmaxcmincmaxZ seg_map_cropZ type_map_cropZ inst_typeZ type_listZ type_pixels type_dict type_probr4r4r5r;ls    zGenerateInstanceType.__call__N)r=r>r?r@rrArBr;r4r4r4r5r)escsReZdZdZdd d d d d dd d dd d ddd fdd ZddddddZZS)r*a The post-processing transform for HoVerNet model to generate instance segmentation map. It generates an instance segmentation map as well as a dictionary containing centroids, bounding boxes, and contours for each instance. Args: activation: the activation layer to be applied on the input probability map. It can be "softmax" or "sigmoid" string, or any callable. Defaults to "softmax". mask_threshold: a float value to threshold to binarize probability map to generate mask. min_object_size: objects smaller than this size (in pixel) are removed. Defaults to 10. sobel_kernel_size: the size of the Sobel kernel used in :py:class:`GenerateInstanceBorder`. Defaults to 5. distance_smooth_fn: smoothing function for distance map. If not provided, :py:class:`monai.transforms.intensity.GaussianSmooth()` will be used. marker_threshold: a float value to threshold to binarize instance border map for markers. It turns uncertain area to 1 and other area to 0. Defaults to 0.4. marker_radius: the radius of the disk-shaped footprint used in `opening` of markers. Defaults to 2. marker_postprocess_fn: post-process function for watershed markers. If not provided, :py:class:`monai.transforms.post.FillHoles()` will be used. watershed_connectivity: `connectivity` argument of `skimage.segmentation.watershed`. min_num_points: minimum number of points to be considered as a contour. Defaults to 3. contour_level: an optional value for `skimage.measure.find_contours` to find contours in the array. If not provided, the level is set to `(max(image) + min(image)) / 2`. device: target device to put the output Tensor data. rENrFrfr}rnr,rkrGrHrIrtr~r-str | torch.device | Noner.) rJmask_thresholdrLsobel_kernel_sizedistance_smooth_fnmarker_threshold marker_radiusmarker_postprocess_fnwatershed_connectivityrrdevicer1c spt| |_t|||d|_t|d|_t|d|_t ||||d|_ t | d|_ t | | d|_t|_dS)N)rJrKrLrh)ru)rKrrrL)r/)rr)superr6rr"generate_watershed_maskr#generate_instance_borderr$generate_distance_mapr%generate_watershed_markersr!rr'generate_instance_contourr(generate_instance_centroid) r3rJrrLrrrrrrrrr __class__r4r5r6s*    z*HoVerNetInstanceMapPostProcessing.__init__rztuple[dict, NdarrayOrTensor])nuclear_predictionrjr1cCs||}|||}|||}|||}||||}tt|dh}i} |D]} || k} t| } | dd| dd| dd| dd| ddf} | dd| ddg} | t | | }|dk rX| | | }| ||d| | <qXt ||j d}| |fS)a"post-process instance segmentation branches (NP and HV) to generate instance segmentation map. Args: nuclear_prediction: the output of NP (nuclear prediction) branch of HoVerNet model hover_map: the output of HV (hover map) branch of HoVerNet model rNr,rnrk) bounding_boxr contourr)rrrrrsetrCrr rr rrr)r3rrjZwatershed_maskZinstance_bordersr|Zwatershed_markers instance_mapZ instance_ids instance_infoinst_id instance_maskZ instance_bboxrZinstance_contourZinstance_centroidr4r4r5r;s6         z*HoVerNetInstanceMapPostProcessing.__call__) rENrFrfNr}rnNr,rkNNr=r>r?r@r6r; __classcell__r4r4rr5r*s,"csDeZdZdZdddddd d fd d Zd dd ddddZZS)r+a# The post-processing transform for HoVerNet model to generate nuclear type information. It updates the input instance info dictionary with information about types of the nuclei (value and probability). Also if requested (`return_type_map=True`), it generates a pixel-level type map. Args: activation: the activation layer to be applied on nuclear type branch. It can be "softmax" or "sigmoid" string, or any callable. Defaults to "softmax". threshold: an optional float value to threshold to binarize probability map. If not provided, defaults to 0.5 when activation is not "softmax", otherwise None. return_type_map: whether to calculate and return pixel-level type map. device: target device to put the output Tensor data. rENTrGrHrrr.)rJrKreturn_type_maprr1cst||_||_t|_d}d}d}t|trj|dkrFd}q|dkrXd}qt d|dn"t |rx|}nt dt |dt |||d|_ |s|dkrd }t||d |_dS) NFrETrMrNrOrPrQrSrT)rr6rrr)generate_instance_typerXrYrZr[r\r]rrJrr^)r3rJrKrrr`rarbrr4r5r6s,      z*HoVerNetNuclearTypePostProcessing.__init__rzdict[int, dict]z#tuple[dict, NdarrayOrTensor | None])type_predictionrrr1cCs||}||}d}|jr4tt|j|d}|D]\}|j||||d|d\}}|||d<|||d<|dk r8||||k<t||j d}q8||fS)aProcess NC (type prediction) branch and combine it with instance segmentation It updates the instance_info with instance type and associated probability, and generate instance type map. Args: instance_info: instance information dictionary, the output of :py:class:`HoVerNetInstanceMapPostProcessing` instance_map: instance segmentation map, the output of :py:class:`HoVerNetInstanceMapPostProcessing` type_prediction: the output of NC (type prediction) branch of HoVerNet model Nrr)rrrrrr]r) rJr^rrtorchzerosrqrrr)r3rrrtype_maprZ instance_typeZinstance_type_probr4r4r5r;#s$       z*HoVerNetNuclearTypePostProcessing.__call__)rENTNrr4r4rr5r+s$)8 __future__rrtypingrrnumpyrCrmonai.config.type_definitionsrrmonai.transformsrrr r r r r monai.transforms.transformr0monai.transforms.utils_pytorch_numpy_unificationrrrrr monai.utilsrrrmonai.utils.miscrmonai.utils.type_conversionrrr_rrrrr __all__r!r"r#r$r%r&r'r(r)r*r+r4r4r4r5 sR $  )GE+G5/!&d