o # i.@sddlmZddlZddlZddlZddlmZddlmZddl m Z ddl m Z m Z ddlmZmZddlmZe dBdCddZe dBdDddZdBddZdEddZ   dFdGd!d"ZdHd'd(Z dIdJd.d/Z  0     dKdLd@dAZdS)M) annotationsN)Sequence)Path)overload)KeysCollectionPathLike)partition_datasetselect_cross_validation_folds) ensure_tupleFbase_dirrelement check_pathboolreturnstrcCdSNr r r rr_/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/data/decathlon_datalist.py _compute_pathrlist[PathLike] list[str]cCrrrrrrrrrcsld fdd t|ttjfr|St|tr4|D]}t|ttjfs)|Sqfdd|DS|S) a^ Args: base_dir: the base directory of the dataset. element: file path(s) to append to directory. check_path: if `True`, only compute when the result is an existing path. Raises: TypeError: When ``element`` contains a non ``str``. TypeError: When ``element`` type is not in ``Union[list, str]``. r ritemcs2tjtj||}rtj|s|S|Sr)ospathnormpathjoinexists)r rresultr rr _join_path/sz!_compute_path.._join_pathcsg|]}|qSrr).0e)r"r rr <sz!_compute_path..N)r rrr) isinstancerrrlist)r r r r$r)r"r r rr"s   is_segmentationitems list[dict]cCsz|D]8}t|tstdt|jd|D]!\}}|dks&|r0|dkr0t||dd||<qt||dd||<qq|S)a( Args: base_dir: the base directory of the dataset. is_segmentation: whether the datalist is for segmentation task. items: list of data items, each of which is a dict keyed by element names. Raises: TypeError: When ``items`` contains a non ``dict``. z+Every item in items must be a dict but got .imagelabelFr!T)r&dict TypeErrortype__name__r)r)r r(r)rkvrrr _append_paths@s r4Ttrainingdata_list_file_path data_list_keyPathLike | NonecCst|}|std|dt| }t|}Wdn1s$wY||vr8td|d|d||}|dkrNt|dtsNd d |D}|durU|j}t |||S) a5 Load image/label paths of decathlon challenge from JSON file JSON file should follow the format of the Medical Segmentation Decathlon datalist.json files, see http://medicaldecathlon.com. The files are structured as follows: .. code-block:: python { "metadata_key_0": "metadata_value_0", "metadata_key_1": "metadata_value_1", ..., "training": [ {"image": "path/to/image_1.nii.gz", "label": "path/to/label_1.nii.gz"}, {"image": "path/to/image_2.nii.gz", "label": "path/to/label_2.nii.gz"}, ... ], "test": [ "path/to/image_3.nii.gz", "path/to/image_4.nii.gz", ... ] } The metadata keys are optional for loading the datalist, but include: - some string items: ``name``, ``description``, ``reference``, ``licence``, ``release``, ``tensorImageSize`` - two dict items: ``modality`` (keyed by channel index), and ``labels`` (keyed by label index) - and two integer items: ``numTraining`` and ``numTest``, with the number of items. The ``training`` key contains a list of dictionaries, each of which has at least the ``image`` and ``label`` keys. The image and label are loaded by :py:func:`monai.transforms.LoadImaged`, so both can be either a single file path or a list of file paths, in which case they are loaded as multi-channel images. Each item can also include a ``fold`` key for cross-validation purposes. The "test" key contains a list of image paths, without labels, MONAI also supports a "validation" list with the same format as the "training" list. Args: data_list_file_path: the path to the json file of datalist. is_segmentation: whether the datalist is for segmentation task, default is True. data_list_key: the key to get a list of dictionary to be used, default is "training". base_dir: the base directory of the dataset, if None, use the datalist directory. Raises: ValueError: When ``data_list_file_path`` does not point to a file. ValueError: When ``data_list_key`` is not specified in the data list file. Returns a list of data items, each of which is a dict keyed by element names, for example: .. code-block:: python [ {'image': '/workspace/data/chest_19.nii.gz', 'label': '/workspace/labels/chest_19.nii.gz'}, {'image': '/workspace/data/chest_31.nii.gz', 'label': '/workspace/labels/chest_31.nii.gz'}, ] zData list file  does not exist.Nz Data list z not specified in "z".testrcSsg|]}d|iqS)r,r)r#irrrr%sz+load_decathlon_datalist..) ris_file ValueErroropenjsonloadr&r.parentr4)r6r(r7r json_file json_dataZ expected_datarrrload_decathlon_datalistWsA   rDdata_property_file_path property_keysSequence[str] | strr.cCst|}|std|dt| }t|}Wdn1s$wYi}t|D]}||vr=td|d||||<q/|S)aExtract the properties with the specified keys from the Decathlon JSON file. See under `load_decathlon_datalist` for the expected keys in the Decathlon challenge. Args: data_property_file_path: the path to the JSON file of data properties. property_keys: expected keys to load from the JSON file, for example, we have these keys in the decathlon challenge: `name`, `description`, `reference`, `licence`, `tensorImageSize`, `modality`, `labels`, `numTraining`, `numTest`, etc. zData property file r9Nzkey z" is not in the data property file.)rr<r=r>r?r@r KeyError)rErFrBrC propertieskeyrrrload_decathlon_propertiess    rKdatalistkeysrroot_dirallow_missing_keysc Csg}|D]R}t|D]K}||vr|std|d|q t||D]1}t|ttjfs8td|d|dt|}t|ttjfrKt||}|sT| |q#q q|S)a|Checks whether some files in the Decathlon datalist are missing. It would be helpful to check missing files before a heavy training run. Args: datalist: a list of data items, every item is a dictionary. usually generated by `load_decathlon_datalist` API. keys: expected keys to check in the datalist. root_dir: if not None, provides the root dir for the relative file paths in `datalist`. allow_missing_keys: whether allow missing keys in the datalist items. if False, raise exception if missing. default to False. Returns: A list of missing filenames. zkey `z#` is missing in the datalist item: zfilepath of key `z2` must be a string or a list of strings, but got: r+) r r=r&rrrrjoinpathrappend)rLrMrNrOZ missing_filesrr2frrrcheck_missing_filess&  rS validationnfoldsint train_foldsSequence[int] | int val_folds train_keyval_keyfilenamePath | str | Noneshuffleseed check_missingKeysCollection | None str | None raise_errorcCs| r| durt|| | | }|rd|}| rt|t|t||||d}t||d}t||d}||||i}t|ttfr^t |d}t j ||ddWd|S1sYwY|S)aC Utility to create new Decathlon style datalist based on cross validation partition. Args: datalist: loaded list of dictionaries for all the items to partition. nfolds: number of the kfold split. train_folds: indices of folds for training part. val_folds: indices of folds for validation part. train_key: the key of train part in the new datalist, defaults to "training". val_key: the key of validation part in the new datalist, defaults to "validation". filename: if not None and ends with ".json", save the new datalist into JSON file. shuffle: whether to shuffle the datalist before partition, defaults to `True`. seed: if `shuffle` is True, set the random seed, defaults to `0`. check_missing: whether to check all the files specified by `keys` are existing. keys: if not None and check_missing_files is True, the expected keys to check in the datalist. root_dir: if not None, provides the root dir for the relative file paths in `datalist`. allow_missing_keys: if check_missing_files is `True`, whether allow missing keys in the datalist items. if False, raise exception if missing. default to False. raise_error: when found missing files, if `True`, raise exception and stop, if `False`, print warning. Nz(some files of the datalist are missing: )datanum_partitionsr^r_) partitionsfoldsw)indent) rSr=warningswarnrr r&rrr>r?dump)rLrUrWrYrZr[r\r^r_r`rMrNrOrcfilesmsgrdZ train_listZval_listretrRrrr create_cross_validation_datalists$ %       rq)F)r rr rr rrr)r rr rr rrr)r rr(rr)r*rr*)Tr5N) r6rr(rr7rr r8rr*)rErrFrGrr.)NF)rLr*rMrrNr8rOr) r5rTNTrFNNFT)rLr*rUrVrWrXrYrXrZrr[rr\r]r^rr_rVr`rrMrarNrbrOrrcr) __future__rr?rrkcollections.abcrpathlibrtypingr monai.configrrmonai.data.utilsrr monai.utilsr rr4rDrKrSrqrrrrsD        S +