o  iH@sddlmZddlZddlZddlZddlZddlZddlmZddl m Z ddl m Z m Z ddlZddlZddlmZddlmZddlmZdd lmZdd lmZdd lmZmZdd lmZm Z m!Z!gd Z"e dde\Z#Z$e d\Z%Z&dbddZ'dcddZ(dddedd Z) ! !dfdgd+d,Z* dddhd1d2Z+didjdAsz&get_foreground_image..T) select_fn allow_smaller)r rnpndarray)rcopperimage_foregroundrrr!r2s rlabelcCst||dk}|S)a Get foreground image pixel values and mask out the non-labeled area. Args image: ndarray image to segment. label: ndarray the image input and annotated with class IDs. Returns: 1D array of foreground image with label > 0 rr )rr)Zlabel_foregroundrrr!rFs rT mask_indexuse_gpubooltuple[list[Any], int]c s|td\}}g}|jjdkrotro|ro|rot|}|j|}t |t |}|D].}t ||k} tj | dd tj| dd fddttD} || q-t|} ~~~~ ~t|| fStrtj|jddd \}} td | d D].}t ||k} tj | dd tj| dd fd dttD} || q|| fStd ) a4 Find all connected components and their bounding shape. Backend can be cuPy/cuCIM or Numpy depending on the hardware. Args: mask_index: a binary mask. use_gpu: a switch to use GPU/CUDA or not. If GPU is unavailable, CPU will be used regardless of this setting. z cucim.skimagecudar)axisc g|] }||dqSr.0iZ comp_idx_maxZ comp_idx_minrr! l z!get_label_ccp..T) background return_numr2cr0r1rr3r6rr!r7yr8zVCannot find one of the following required dependencies: {cuPy+cuCIM} or {scikit-image})rdevicetypehas_cprshortmeasurer)cpuniquenonzeroargwheremintolistmaxrangelenappendget_default_memory_poolfree_all_blocks has_measure measure_npdatacpunumpyr% RuntimeError) r*r+skimage has_cucim shape_listZ mask_cupyZlabeledvalsZncompZcomp_idxZ bbox_shape ncomponentsrr6r!rVs4       rF data_list list[dict] fixed_keyslist[str | int]ragged bool | None allow_missingkwargsrc Ks@g}|D]q}t|}t|D] \}} t| ||<q|t|} | dur4|r-|dqt|dt| t rB|t | qt| t j tfrT|| qt| t jr`|| qt| trn|t | qt| jd|rdd|D}t|dkrt dgS|rt j|fi|St j|gfi|S)a Get the nested value in a list of dictionary that shares the same structure. Args: data_list: a list of dictionary {key1: {key2: np.ndarray}}. fixed_keys: a list of keys that records to path to the value in the dict elements. ragged: if True, numbers can be in list of lists or ragged format so concat mode needs change. allow_missing: if True, it will return a None if the value cannot be found. Returns: nd.array of concatenated array. Nz is not nested in the dictionaryz concat is not supported.cSsg|]}|dur|qSNr)r4r rrr!r7sz$concat_val_to_np..r)r enumeratestrgetr joinrJAttributeError isinstancelistr%arraytorchTensorr rPrQr&rNotImplementedError __class__rI concatenate) rXrZr\r^r_Znp_listrOparserr5keyvalrrr!rs4       rkeys list[str] zero_insertdict[str, np.ndarray]c KsHi}|D]}|r d|gn|g}t|||fi|}|||iq|S)a Get the nested value in a list of dictionary that shares the same structure iteratively on all keys. It returns a dictionary with keys with the found values in nd.ndarray. Args: data_list: a list of dictionary {key1: {key2: np.ndarray}}. fixed_keys: a list of keys that records to path to the value in the dict elements. keys: a list of string keys that will be iterated to generate a dict output. zero_insert: insert a zero in the list so that it can find the value in element 0 before getting the keys flatten: if True, numbers are flattened before concat. Returns: a dict with keys - nd.array of concatenated array pair. r)rupdate) rXrZrqrsr_ret_dictroZaddonrprrr!rs rtrainingdatalist str | dictbasedirrbfoldintrotuple[list, list]c st|tr t|}n|}t||}|D]>}|D]7\}}t||tr4fdd||D||<qt||trRt||dkrLtj ||n||||<qqg} g} |D]}d|vrl|d|krl| |qZ| |qZ| | fS)a Read a list of data dictionary `datalist` Args: datalist: the name of a JSON file listing the data, or a dictionary. basedir: directory of image files. fold: which fold to use (0..1 if in training set). key: usually 'training' , but can try 'validation' or 'testing' to get the list data without labels (used in challenges). Returns: A tuple of two arrays (training, validation). csg|] }tj|qSr)ospathrd)r4ivrzrr!r7sz!datafold_read..rr{) rfrbrload_config_fileritemsrgrIr~rrdrJ) rxrzr{ro json_dataZ dict_datadk_trrprrr!rs&   .  rreportdict report_formatcCs|D]=\}}||vrdS||}t|trAt|trAt|dkr'tdt|dkr>t|dkr>t|d|dSdSqdS)z Compares the report and the report_format that has only keys. Args: report: dict that has real values. report_format: dict that only has keys and list-nested value. Fr2z%list length in report_format is not 1rT)rrfrgrI UserWarningr)rrZk_fmtZv_fmtvrrr!rs ralgor template_pathPathLike | Nonealgo_meta_datac Kst|t|d}tj|d}|D] \}}|||iqt|}t |d}| |Wd|S1s>wY|S)a Export the Algo object to pickle file. Args: algo: Algo-like object. template_path: a str path that is needed to be added to the sys.path to instantiate the class. algo_meta_data: additional keyword to save into the dictionary, for example, model training info such as acc/best_metrics Returns: filename of the pickled Algo object ) algo_bytesrzalgo_object.pklwbN) pickledumpsrbr~rrdget_output_pathrruopenwrite) rrrrO pkl_filenamerr data_bytesf_pirrr!rs     rrc Kslt|d }|}Wdn1swYt|}t|ts,td|jdd|vr8td|d|d}|dd}g}t j t |rh| t j t || t j t j t |d t j t |r| t j || t j t j |d t j |} t j | d d } t j | r| t j | t|d krt|} d| _nMt|D]E\} } ztj | t|} Wn3ty}z'td | d tj | t|dkrtd|d||WYd}~qd}~ww| | _t j | t j | krt| d| d| | _i}|D] \}}|||iq%| |fS)a Import the Algo object from a pickle file. Args: pkl_filename: the name of the pickle file. template_path: a folder containing files to instantiate the Algo. Besides the `template_path`, this function will also attempt to use the `template_path` saved in the pickle file and a directory named `algorithm_templates` in the parent folder of the folder containing the pickle file. Returns: algo: the Algo object saved in the pickle file. algo_meta_data: additional keyword saved in the pickle file, for example, acc/best_metrics. Raises: ValueError if the pkl_filename does not contain a dict, or the dict does not contain `algo_bytes`. ModuleNotFoundError if it is unable to instantiate the Algo class. rbNzthe data object is z. Dict is expected.rzkey [algo_bytes] not found in z. Unable to instantiate.rz..algorithm_templatesrzFolder z; doesn't contain the Algo templates for Algo instantiation.r2zFailed to instantiate z with z5 is changed. Now override the Algo output_path with: .)rreadrloadsrfr ValueErrorrlpopr~risdirrbrJabspathrddirnamerIrrasysModuleNotFoundErrorloggingdebugr output_pathrru)rrr_rrrOrZalgo_template_pathZtemplate_paths_candidatesZpkl_dirZalgo_template_path_fuzzyrr5pZ not_found_errrrrrrr!r,sb               rargsrgcCs ddd|D}d|dS)z Convert a list of arguments to a string that can be used in python-fire. Args: args: the list of arguments. Returns: the string that can be used in python-fire. ,cSsg|]}t|qSr)rb)r4argrrr!r7sz/list_to_python_fire_arg_str..')rd)rargs_strrrr!list_to_python_fire_arg_strzs rparamscCsRd}|D] \}}t|trtdt|trt|}|d|d|7}q|S)z;convert `params` into '--key_1=value_1 --key_2=value_2 ...'zNested dict is not supported.z --=)rrfrrrgr)rZ cmd_mod_optrrrrr!check_and_set_optional_argss  rcmd cmd_prefix str | NonecKs:|}|r d|vr d}|ds|d7}||t|S)a Prepare the command for subprocess to run the script with the given arguments. Args: cmd: the command or script to run in the distributed job. cmd_prefix: the command prefix to run the script, e.g., "python", "python -m", "python3", "/opt/conda/bin/python3.9 ". kwargs: the keyword arguments to be passed to the script. Returns: the command to run with ``subprocess``. Examples: To prepare a subprocess command "python train.py run -k --config 'a,b'", the function can be called as - _prepare_cmd_default("train.py run -k", config=['a','b']) - _prepare_cmd_default("train.py run -k --config 'a,b'") Nonepython )copyendswithr)rrr_rrrr!_prepare_cmd_defaults   rcKs|}|t|S)a Prepare the command for multi-gpu/multi-node job execution using torchrun. Args: cmd: the command or script to run in the distributed job. kwargs: the keyword arguments to be passed to the script. Returns: the command to append to ``torchrun`` Examples: For command "torchrun --nnodes=1 --nproc_per_node=8 train.py run -k --config 'a,b'", it only prepares command after the torchrun arguments, i.e., "train.py run -k --config 'a,b'". The function can be called as - _prepare_cmd_torchrun("train.py run -k", config=['a','b']) - _prepare_cmd_torchrun("train.py run -k --config 'a,b'") )rr)rr_rrrr!_prepare_cmd_torchruns rcKst|fd|i|S)a Prepare the command for distributed job running using bcprun. Args: script: the script to run in the distributed job. cmd_prefix: the command prefix to run the script, e.g., "python". kwargs: the keyword arguments to be passed to the script. Returns: The command to run the script in the distributed job. Examples: For command "bcprun -n 2 -p 8 -c python train.py run -k --config 'a,b'", it only prepares command after the bcprun arguments, i.e., "train.py run -k --config 'a,b'". the function can be called as - _prepare_cmd_bcprun("train.py run -k", config=['a','b'], n=2, p=8) - _prepare_cmd_bcprun("train.py run -k --config 'a,b'", n=2, p=8) r)r)rrr_rrr!_prepare_cmd_bcprunsrsubprocess.CompletedProcesscKsx|}|}dg}ddg}|D]}||vrtd|d|d|t||g7}q||7}t|fddi|S) a Run the command with torchrun. Args: cmd: the command to run. Typically it is prepared by ``_prepare_cmd_torchrun``. kwargs: the keyword arguments to be passed to the ``torchrun``. Return: the return code of the subprocess command. Ztorchrunnnodesnproc_per_nodeMissing required argument z for torchrun.z--run_cmd_verboseT)rsplitrrbrr)rr_rcmd_listZ torchrun_list required_argsrrrr!_run_cmd_torchruns rcKsv|}dg}ddg}|D]}||vrtd|d|d|t||g7}q |d|gt|fdd i|S) a Run the command with bcprun. Args: cmd: the command to run. Typically it is prepared by ``_prepare_cmd_bcprun``. kwargs: the keyword arguments to be passed to the ``bcprun``. Returns: the return code of the subprocess command. bcprunnrrz for bcprun.-z-crT)rrrbrextendr)rr_rrrrrrr!_run_cmd_bcpruns r)rr rr)rr r)r rr )T)r*r r+r,rr-)FF) rXrYrZr[r\r]r^r]r_rrr) rXrYrZr[rqrrrsr,r_rrrt)rrw) rxryrzrbr{r|rorbrr})rrrrrr,r`)rrrrrrrrb)rrbrrr_rrr)rrgrrb)rrrrb)rrbrrr_rrrb)rrbr_rrrb)rrbr_rrr)7 __future__rrr~r subprocessrrrnumbersrtypingrrrQr%riZmonai.auto3dsegrmonai.bundle.config_parserrmonai.bundle.utilsr monai.configr monai.data.meta_tensorr monai.transformsr r monai.utilsrrr__all__rNrMrAr>rrrrrrrrrrrrrrrrrrrr!sP            . :  '   N