o # iL@s\ddlmZddlZddlZddlmZmZmZmZm Z m Z m Z ddl m Z ddlmZddlmZddlmZddlZddlZddlmZdd lmZdd lmZdd lmZmZdd l m!Z!m"Z"m#Z#m$Z$m%Z%dd l&m'Z'm(Z(m)Z)m*Z*m+Z+erddl,m,Z,dZ-n e+dde*d\Z,Z-gdZ.GdddZ/GdddZ0GdddeZ1GdddeZ2dS)) annotationsN)Callable GeneratorHashableIterableIteratorMappingSequence)deepcopy) ListProxy) ThreadPool) TYPE_CHECKING)KeysCollection) NdarrayTensor)IterableDataset) iter_patchpickle_hashing)ComposeRandomizableTrait Transformapply_transformconvert_to_contiguous) NumpyPadMode ensure_tuplefirst min_versionoptional_import)tqdmTrz4.47.0) PatchDatasetGridPatchDataset PatchIter PatchIterdc@s,eZdZdZdejfdd d ZdddZdS)r z Return a patch generator with predefined properties such as `patch_size`. Typically used with :py:class:`monai.data.GridPatchDataset`. patch_size Sequence[int] start_posmode str | Nonepad_optsdictcKs(dt||_t||_||_||_dS)a Args: patch_size: size of patches to generate slices for, 0/None selects whole dimension start_pos: starting position in the array, default is 0 for each dimension mode: available modes: (Numpy) {``"constant"``, ``"edge"``, ``"linear_ramp"``, ``"maximum"``, ``"mean"``, ``"median"``, ``"minimum"``, ``"reflect"``, ``"symmetric"``, ``"wrap"``, ``"empty"``} (PyTorch) {``"constant"``, ``"reflect"``, ``"replicate"``, ``"circular"``}. One of the listed string values or a user supplied function. If None, no wrapping is performed. Defaults to ``"wrap"``. See also: https://numpy.org/doc/stable/reference/generated/numpy.pad.html https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html requires pytorch >= 1.10 for best compatibility. pad_opts: other arguments for the `np.pad` function. note that `np.pad` treats channel dimension as the first dimension. Note: The `patch_size` is the size of the patch to sample from the input arrays. It is assumed the arrays first dimension is the channel dimension which will be yielded in its entirety so this should not be specified in `patch_size`. For example, for an input 3D array with 1 channel of size (1, 20, 20, 20) a regular grid sampling of eight patches (1, 10, 10, 10) would be specified by a `patch_size` of (10, 10, 10). NN)tupler#rr%r&r()selfr#r%r&r(r"r"Y/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/data/grid_dataset.py__init__1s  zPatchIter.__init__arrayrreturn7Generator[tuple[NdarrayTensor, np.ndarray], None, None]ccs0t|f|j|jdd|jd|jEdHdS)zO Args: array: the image to generate patches from. gF)r#r%overlap copy_backr&N)rr#r%r&r()r,r/r"r"r-__call__UszPatchIter.__call__N)r#r$r%r$r&r'r(r))r/rr0r1)__name__ __module__ __qualname____doc__rWRAPr.r4r"r"r"r-r *s   $r c@s8eZdZdZdZdZdZdejfdd d Z dddZ dS)r!aA Dictionary-based wrapper of :py:class:`monai.data.PatchIter`. Return a patch generator for dictionary data and the coordinate, Typically used with :py:class:`monai.data.GridPatchDataset`. Suppose all the expected fields specified by `keys` have same shape. Args: keys: keys of the corresponding items to iterate patches. patch_size: size of patches to generate slices for, 0/None selects whole dimension start_pos: starting position in the array, default is 0 for each dimension mode: available modes: (Numpy) {``"constant"``, ``"edge"``, ``"linear_ramp"``, ``"maximum"``, ``"mean"``, ``"median"``, ``"minimum"``, ``"reflect"``, ``"symmetric"``, ``"wrap"``, ``"empty"``} (PyTorch) {``"constant"``, ``"reflect"``, ``"replicate"``, ``"circular"``}. One of the listed string values or a user supplied function. If None, no wrapping is performed. Defaults to ``"wrap"``. See also: https://numpy.org/doc/stable/reference/generated/numpy.pad.html https://pytorch.org/docs/stable/generated/torch.nn.functional.pad.html requires pytorch >= 1.10 for best compatibility. pad_opts: other arguments for the `np.pad` function. note that `np.pad` treats channel dimension as the first dimension. Z patch_coordsoriginal_spatial_shaper%r"keysrr#r$r&r'cKs&t||_td|||d||_dS)N)r#r%r&r")rr;r patch_iter)r,r;r#r%r&r(r"r"r-r.s zPatchIterd.__init__data Mapping[Hashable, NdarrayTensor]r0JGenerator[tuple[Mapping[Hashable, NdarrayTensor], np.ndarray], None, None]c#st|tjjdd}tfddjDD]@}|dd}ddtj|D}ttjD] }t|||<q=||j<||j <j j |j <||fVqdS)Ncsg|] }|qSr")r<).0keydr,r"r- z'PatchIterd.__call__..rcSsi|] \}}||dqS)rr")rAkvr"r"r- rFz'PatchIterd.__call__..) r)rr;shapezipset differencer coords_keyoriginal_spatial_shape_keyr<r% start_pos_key)r,r=r:patchcoordsretrGr"rCr-r4s    zPatchIterd.__call__N)r;rr#r$r%r$r&r')r=r>r0r?) r5r6r7r8rNrOrPrr9r.r4r"r"r"r-r!fs  r!c sneZdZdZdddejdddddef d,fdd Zd-d d!Zd.d/d#d$Z d0d&d'Z d(d)Z fd*d+Z Z S)1ra Yields patches from data read from an image dataset. Typically used with `PatchIter` or `PatchIterd` so that the patches are chosen in a contiguous grid sampling scheme. .. code-block:: python import numpy as np from monai.data import GridPatchDataset, DataLoader, PatchIter, RandShiftIntensity # image-level dataset images = [np.arange(16, dtype=float).reshape(1, 4, 4), np.arange(16, dtype=float).reshape(1, 4, 4)] # image-level patch generator, "grid sampling" patch_iter = PatchIter(patch_size=(2, 2), start_pos=(0, 0)) # patch-level intensity shifts patch_intensity = RandShiftIntensity(offsets=1.0, prob=1.0) # construct the dataset ds = GridPatchDataset(data=images, patch_iter=patch_iter, transform=patch_intensity) # use the grid patch dataset for item in DataLoader(ds, batch_size=2, num_workers=2): print("patch size:", item[0].shape) print("coordinates:", item[1]) # >>> patch size: torch.Size([2, 1, 2, 2]) # coordinates: tensor([[[0, 1], [0, 2], [0, 2]], # [[0, 1], [2, 4], [0, 2]]]) Args: data: the data source to read image data from. patch_iter: converts an input image (item from dataset) into a iterable of image patches. `patch_iter(dataset[idx])` must yield a tuple: (patches, coordinates). see also: :py:class:`monai.data.PatchIter` or :py:class:`monai.data.PatchIterd`. transform: a callable data transform operates on the patches. with_coordinates: whether to yield the coordinates of each patch, default to `True`. cache: whether to use cache mache mechanism, default to `False`. see also: :py:class:`monai.data.CacheDataset`. cache_num: number of items to be cached. Default is `sys.maxsize`. will take the minimum of (cache_num, data_length x cache_rate, data_length). cache_rate: percentage of cached data in total, default is 1.0 (cache all). will take the minimum of (cache_num, data_length x cache_rate, data_length). num_workers: the number of worker threads if computing cache in the initialization. If num_workers is None then the number returned by os.cpu_count() is used. If a value less than 1 is specified, 1 will be used instead. progress: whether to display a progress bar. copy_cache: whether to `deepcopy` the cache content before applying the random transforms, default to `True`. if the random transforms don't modify the cached content (for example, randomly crop from the cached image and deepcopy the crop region) or if every cache item is only used once in a `multi-processing` environment, may set `copy=False` for better performance. as_contiguous: whether to convert the cached NumPy array or PyTorch tensor to be contiguous. it may help improve the performance of following logic. hash_func: a callable to compute hash from data items to be cached. defaults to `monai.data.utils.pickle_hashing`. NTFg?r@r=Iterable | Sequencer<r transformCallable | Nonewith_coordinatesboolcache cache_numint cache_ratefloat num_workers int | Noneprogress copy_cache as_contiguous hash_funcCallable[..., bytes]r0Nonec stj|dd|durt|tst|}||_||_||_||_||_| |_ | |_ | |_ | |_ ||_ |j durAtt|j d|_ g|_g|_||_d|_|jdur[|jdd|_|jrnt|trgtd||dSdS)Nr=rUr@cSst|tp t|t Sr*) isinstancerr)tr"r"r-sz+GridPatchDataset.__init__..z+Data can not be iterator when cache is True)superr.rgrr<patch_transformrWset_numset_rater`rarbrcr^maxr[_cache _cache_otherrY first_randomget_index_of_firstr TypeErrorset_data) r,r=r<rUrWrYrZr\r^r`rarbrc __class__r"r-r.s8   zGridPatchDataset.__init__r cs|_fddtjD}ttjtt|jt|_t|dj_ t| dj}t |\_ _dS)aA Set the input data and run deterministic transforms to generate cache content. Note: should call this func after an entire epoch and must set `persistent_workers=False` in PyTorch DataLoader, because it needs to create new worker processes based on new generated cache content. csi|] \}}||qSr")rc)rAirHr,r"r-rIsz-GridPatchDataset.set_data..N)r= enumerateminr[rllenrmrZlist _hash_keysvaluesrK _fill_cacherorp)r,r=mappingindicesr"rxr-rt s  &zGridPatchDataset.set_datar|cCs|jdkrgS|durtt|j}|jrtstd|jr#tr#tndd}t|j }t|| |j |t |ddWdS1sGwYdS)z Compute and fill the cache content from data source. Args: indices: target indices in the `self.data` source to compute cache. if None, use the first `cache_num` items. rNz>tqdm is not installed, will not show the caching progress bar.c[s|Sr*r")rH_r"r"r-ri+sz.GridPatchDataset._fill_cache..zLoading dataset)totaldesc) rZr|ranger`has_tqdmwarningswarnrr r^imap_load_cache_itemr{)r,rZpfuncpr"r"r-rs    $zGridPatchDataset._fill_cacheidxcCs|j|}gg}}||D]1^}}|jdur!|j||jdd}|jr+t|tjd}|jr;t |dkr;| |d| |q||fS)zN Args: idx: the index of the input data sequence. NT)end threading) memory_formatr) r=r<rqrkrbrtorchcontiguous_formatrWr{append)r,ritemZ patch_cacheZ other_cacherQothersr"r"r-r/s    z!GridPatchDataset._load_cache_itemcks\|D](^}}|}|jdur|j|fi|}|jr(t|dkr(||dfVq|VqdS)z yield patches optionally post-processed by transform. Args: src: a iterable of image patches. apply_args: other args for `self.patch_transform`. Nr)rkrWr{)r,srcZ apply_argsrQr out_patchr"r"r-_generate_patchesAs  z"GridPatchDataset._generate_patchesc#s|jrZd}tD]L}||}||jvr|j|}|dur-|||EdHq |jdur6t d|j|}|j |}|j rGt |n|}|jt |||jdEdHq dStD] }|||EdHq_dS)NzNCache buffer is not initialized, please call `set_data()` before epoch begins.)start)rYrj__iter__rcr}indexrr<ro RuntimeErrorrprar rKrq)r, cache_indeximagerBr=otherrur"r-rSs*      zGridPatchDataset.__iter__)r=rTr<rrUrVrWrXrYrXrZr[r\r]r^r_r`rXrarXrbrXrcrdr0re)r=r r0rer*)r0r|)rr[)r5r6r7r8sysmaxsizerr.rtrrrr __classcell__r"r"rur-rs$@ ,  rcs<eZdZdZ ddfdd ZdddZfddZZS)ra Yields patches from data read from an image dataset. The patches are generated by a user-specified callable `patch_func`, and are optionally post-processed by `transform`. For example, to generate random patch samples from an image dataset: .. code-block:: python import numpy as np from monai.data import PatchDataset, DataLoader from monai.transforms import RandSpatialCropSamples, RandShiftIntensity # image dataset images = [np.arange(16, dtype=float).reshape(1, 4, 4), np.arange(16, dtype=float).reshape(1, 4, 4)] # image patch sampler n_samples = 5 sampler = RandSpatialCropSamples(roi_size=(3, 3), num_samples=n_samples, random_center=True, random_size=False) # patch-level intensity shifts patch_intensity = RandShiftIntensity(offsets=1.0, prob=1.0) # construct the patch dataset ds = PatchDataset(dataset=images, patch_func=sampler, samples_per_image=n_samples, transform=patch_intensity) # use the patch dataset, length: len(images) x samplers_per_image print(len(ds)) >>> 10 for item in DataLoader(ds, batch_size=2, shuffle=True, num_workers=2): print(item.shape) >>> torch.Size([2, 1, 3, 3]) r@Nr=r patch_funcrsamples_per_imager[rUrVr0recs:tj|dd||_|dkrtdt||_||_dS)a Args: data: an image dataset to extract patches from. patch_func: converts an input image (item from dataset) into a sequence of image patches. patch_func(dataset[idx]) must return a sequence of patches (length `samples_per_image`). samples_per_image: `patch_func` should return a sequence of `samples_per_image` elements. transform: transform applied to each patch. Nrfrz-sampler_per_image must be a positive integer.)rjr.r ValueErrorr[rrk)r,r=rrrUrur"r-r.s    zPatchDataset.__init__cCst|j|jSr*)r{r=rrxr"r"r-__len__szPatchDataset.__len__c#sntD].}||}t||jkrtd|jd|D]}|}|jdur0t|j|dd}|VqqdS)NzA`patch_func` must return a sequence of length: samples_per_image=.F) map_items)rjrrr{rRuntimeWarningrkr)r,rpatchesrQrrur"r-rs   zPatchDataset.__iter__)r@N) r=r rrrr[rUrVr0re)r0r[)r5r6r7r8r.rrrr"r"rur-rns ) r)3 __future__rrrcollections.abcrrrrrrr copyr multiprocessing.managersr multiprocessing.poolr typingr numpynpr monai.configrmonai.config.type_definitionsrZmonai.data.iterable_datasetrmonai.data.utilsrrmonai.transformsrrrrr monai.utilsrrrrrrr__all__r r!rrr"r"r"r-s4 $        <:O