U Ph:@sddlmZddlZddlZddlZddlZddlZddlmZm Z ddl m Z ddl m Z ddlmZddlmZddlmZmZddlZdd lmZdd lmZdd lmZmZmZmZdd lm Z dd l!m"Z"ddl#m$Z$m%Z%ddl&m'Z'm(Z(ddl)m*Z*m+Z+ddl,m-Z-m.Z.ddl/m0Z0m1Z1m2Z2m3Z3m4Z4m5Z5ddl6m7Z7m8Z8m9Z9m:Z:m;Z;mZ?ed?d@ZTd/d/d/dAdBdCZUd/d/d/dAdDdEZVdd/dFd/d%d0dGdHdIZWdFd/d/d%d0dJdKdLZXdd/d/d/dNdOdPZYdd/d/d/dNdQdRZZdFd/d/dSd%d0dTdUdVZ[dWdXZ\d/d/d/dYdZd[d\Z]dd]dFd^d_d`Z^dddeLdddMd#df dSdSd]d/dSdSdSd%dSd0da dbdcZ_e8dddedfdgdhe8didedfdgdhe8djdedfdkdddlddmdeLddMd#dddndddd#ddfd/dodSd/dSd%d]d/dSdSd%dSdSdpdqdSdrd%drdSd&dsdtdudvZ`dd/d/dSdydzd{d|Zadd/d/dSd}dzd~dZbdd/d/d/dSddddZcdd/dSd/d/dSddddZdddSdSdSdddSd$dSd&d0d ddZeddqdSd&d0dddZfddd]ddSdSdSd&d0dddZgddd/d+dddZhdd+dddZiddSdddSddddrdSd&d0d ddZjddd/d/d/d/d/d&d0d ddZkddSd]dSdddSdddSdd&d0d ddZlddSd]dSdddSdddSdd&d0d ddZmddSd]dSdddSdSddddddddSdd&d0dddZnddd]dod%ddd0dddZoddZpdd/d/d/dSddSdd&d&d ddZqddqddSd&d&dddZrddSdSd0dddZsdS)) annotationsN)MappingSequence)Path)locate)copyfile)dedent)AnyCallable) is_available)_get_all_ngc_models) _basename download_url extractall get_logger)ConfigComponent) ConfigParser)DEFAULT_INFERENCEDEFAULT_METADATA)BundleWorkflowConfigWorkflow) IgniteInfoPathLike)load_net_with_metadatasave_net_with_metadata)convert_to_onnxconvert_to_torchscriptconvert_to_trtcopy_model_stateget_state_dict save_state)check_parent_dirdeprecated_arg ensure_tupleget_equivalent_dtype min_versionoptional_import pprint_edgesZ jsonschemavalidatenamezjsonschema.exceptionsValidationErrorzignite.handlers Checkpointrequestsgetonnxhuggingface_hub) module_nameZBUNDLE_DOWNLOAD_SRC monaihostingTzstr | dict | Noneboolr dict)args ignore_nonekwargsreturncKst|tr|ni}t|tr&t|}t|ttfrftdd|Drf|\}}t||ft||f|}t|tst|S| D]P\}}|r|dkrq|t|trt| |trt|||f|||<q||||<q||S)a% Update the `args` dictionary with the input `kwargs`. For dict data, recursively update the content based on the keys. Example:: from monai.bundle import update_kwargs update_kwargs({'exist': 1}, exist=2, new_arg=3) # return {'exist': 2, 'new_arg': 3} Args: args: source `args` dictionary (or a json/yaml filename to read as dictionary) to update. ignore_none: whether to ignore input args with None value, default to `True`. kwargs: key=value pairs to be merged into `args`. css|]}t|tVqdSN) isinstancestr).0xr?I/home/dell461/cl/sdc2/HISourceFinder-master-l/src/monai/bundle/scripts.py [sz update_kwargs..N) r;r5r<rload_config_filetuplelistall update_kwargsitemsr.)r6r7r8args_primary overrideskvr?r?r@rFFs      rFrC)srcr6r8r9cs,tfdd|Dfdd|DS)z{ Pop args from the `src` dictionary based on specified keys in `args` and (key, default value) pairs in `kwargs`. csg|]}|qSr?pop)r=irMr?r@ ssz_pop_args..csg|]\}}||qSr?rNr=rKrLrQr?r@rRss)rCrG)rMr6r8r?rQr@ _pop_argsnsrTr<None)tagr6r9c CsNtd|d|D]$\}}td|dt|tqtddS)Nz*--- input summary of monai.bundle.scripts.z ---z> z: z--- )loggerinforGr'PPRINT_CONFIG_N)rVr6r*valr?r?r@_log_input_summaryvsr[z list[str])exprr9cCst|}ddt|DS)z Parse the expression and discover what variables are present in it based on ast module. Args: expr: source expression to parse. cSsg|]}t|tjr|jqSr?)r;astNameid)r=mr?r?r@rRs z"_get_var_names..)r]parsewalk)r\treer?r?r@_get_var_names}s rdzSequence[str | int]int)shapepnanyr9cCsg}|D]}t|tr"||qt|tr~|dkr@||qt|D]}|dkrHtd|dqH|t|||dqtdt|d|dqt|S)a2 Get spatial shape for fake data according to the specified shape pattern. It supports `int` number and `string` with formats like: "32", "32 * n", "32 ** p", "32 ** p *n". Args: shape: specified pattern for the spatial shape. p: power factor to generate fake data shape if dim of expected shape is "x**p", default to 1. p: multiply factor to generate fake data shape if dim of expected shape is "x*n", default to 1. any: specified size to generate fake data shape if dim of expected shape is "*", default to 1. *)rhriz4only support variables 'p' and 'n' so far, but got: .z4spatial shape items must be int or string, but got:  ) r;rfappendr<rd ValueErrorevaltyperC)rgrhrirjretrPcr?r?r@_get_fake_spatial_shapes      rt) repo_owner repo_nametag_namefilenamer9cCsd|d|d|d|S)Nzhttps://github.com//z/releases/download/r?)rurvrwrxr?r?r@_get_git_release_urlsrz) model_nameversionr9cCsd|d|dS)Nz9https://api.ngc.nvidia.com/v2/models/nvidia/monaitoolkit/ /versions/z/ziplowerr{r|r?r?r@_get_ngc_bundle_urlsrc Cs*d}|d|d|d|d|d S)N8https://api.ngc.nvidia.com/v2/models/nvidia/monaihostingryr}z/files/_v.zipr~)r{r|Zmonaihosting_root_pathr?r?r@_get_monaihosting_bundle_urlsrr)repo download_pathrxprogressr9c Cs\|d\}}}d|kr |d7}t||||d}||}t||d|dt||dddS)Nryr)rwrxurlfilepathhash_valrTr output_dirhas_base)splitrzrr) rrrxrrurvrwrrr?r?r@_download_from_githubs r)rrxr|rr9cCsBt||d}||d|d}t||d|dt||dddS)NrrrrTr)rrr)rrxr|rrrr?r?r@_download_from_monaihostings rmonai_)r*prefixr9cCs||r|S||Sr:) startswithr*rr?r?r@_add_ngc_prefixs rcCs||r|t|dS|Sr:)rlenrr?r?r@_remove_ngc_prefixs rz str | None)rrxr| remove_prefixrr9cCsdt|}t||d}||d|d}|r8t||d}||}t||d|dt||dddS)NrrrrrTr)rrrrr)rrxr|rrrr extract_pathr?r?r@_download_from_ngcs   rcCs\d}|d|}tddd\}}|r<||}|ntdt|j}|ddS) Nrryr-r.r)z6NGC API requires requests package. Please install it.modellatestVersionIdStr)rr&raise_for_statusrojsonloadstext)r*rfull_url requests_get has_requestsresp model_infor?r?r@'_get_latest_bundle_version_monaihostings  rz'dict[str, list[str] | str] | Any | None)sourcer*rr9c Cs|dkrBt|}t|}|D]}|d|kr |dSq dS|dkrRt|S|dkr|d\}}}t||d||ddS|d krtj|d }t|j d krd d |j D} d| krdn| dgd } ndd |j Dd } | St d|ddS)Nngcr*latestr2githubry)rrVlatest_versionr0)repo_idrcSsg|] }|jqSr?r))r=tr?r?r@rRsz._get_latest_bundle_version..cSsg|] }|jqSr?r))r=br?r?r@rRszZTo get the latest bundle version, source should be 'github', 'monaihosting' or 'ngc', got rl) rr valuesrrget_bundle_versionsr0list_repo_refsrtagsZbranchesro) rr*r model_dictrLrurvrwrefs all_versionsrr?r?r@_get_latest_bundle_versions,    rzPathLike | None) bundle_dirr9cCs<|dkr4tddd\}}|r,t|d}ntdt|S)Nz torch.hubget_dirr)bundlezVbundle_dir=None, but no suitable default directory computed. Upgrade Pytorch to 1.6+ ?)r&rro)rrhas_homer?r?r@_process_bundle_dirs r) r*r|rrrrrr args_filer9c  Cst|||||||||d } td| dt| ddddddddd \} } } } }}}}t|}| dkrdd} t| d d kr| d krtd n"t| d d kr| d krtd|dk r|dk r||d}n|t|}t||d| dt ||ddn|dkrtd| d|dkr*t | || d}| dkr^|dk rLd ||g}t | ||| dnt| dkrzt |||| dnX| dkrt|||| | dn:| d krtj ||}tj| ||dntd| ddS) a download bundle from the specified source or url. The bundle should be a zip file and it will be extracted after downloading. This function refers to: https://pytorch.org/docs/stable/_modules/torch/hub.html Typical usage examples: .. code-block:: bash # Execute this module as a CLI entry, and download bundle from the model-zoo repo: python -m monai.bundle download --name --version "0.1.0" --bundle_dir "./" # Execute this module as a CLI entry, and download bundle from specified github repo: python -m monai.bundle download --name --source "github" --repo "repo_owner/repo_name/release_tag" # Execute this module as a CLI entry, and download bundle from ngc with latest version: python -m monai.bundle download --name --source "ngc" --bundle_dir "./" # Execute this module as a CLI entry, and download bundle from monaihosting with latest version: python -m monai.bundle download --name --source "monaihosting" --bundle_dir "./" # Execute this module as a CLI entry, and download bundle from Hugging Face Hub: python -m monai.bundle download --name "bundle_name" --source "huggingface_hub" --repo "repo_owner/repo_name" # Execute this module as a CLI entry, and download bundle via URL: python -m monai.bundle download --name --url # Set default args of `run` in a JSON / YAML file, help to record and simplify the command line. # Other args still can override the default args at runtime. # The content of the JSON / YAML file is a dictionary. For example: # {"name": "spleen", "bundle_dir": "download", "source": ""} # then do the following command for downloading: python -m monai.bundle download --args_file "args.json" --source "github" Args: name: bundle name. If `None` and `url` is `None`, it must be provided in `args_file`. for example: "spleen_ct_segmentation", "prostate_mri_anatomy" in model-zoo: https://github.com/Project-MONAI/model-zoo/releases/tag/hosting_storage_v1. "monai_brats_mri_segmentation" in ngc: https://catalog.ngc.nvidia.com/models?filters=&orderBy=scoreDESC&query=monai. version: version name of the target bundle to download, like: "0.1.0". If `None`, will download the latest version (or the last commit to the `main` branch in the case of Hugging Face Hub). bundle_dir: target directory to store the downloaded data. Default is `bundle` subfolder under `torch.hub.get_dir()`. source: storage location name. This argument is used when `url` is `None`. In default, the value is achieved from the environment variable BUNDLE_DOWNLOAD_SRC, and it should be "ngc", "monaihosting", "github", or "huggingface_hub". repo: repo name. This argument is used when `url` is `None` and `source` is "github" or "huggingface_hub". If `source` is "github", it should be in the form of "repo_owner/repo_name/release_tag". If `source` is "huggingface_hub", it should be in the form of "repo_owner/repo_name". url: url to download the data. If not `None`, data will be downloaded directly and `source` will not be checked. If `name` is `None`, filename is determined by `monai.apps.utils._basename(url)`. remove_prefix: This argument is used when `source` is "ngc". Currently, all ngc bundles have the ``monai_`` prefix, which is not existing in their model zoo contrasts. In order to maintain the consistency between these two sources, remove prefix is necessary. Therefore, if specified, downloaded folder name will remove the prefix. progress: whether to display a progress bar. args_file: a JSON or YAML file to provide default values for all the args in this function. so that the command line inputs can be simplified. ) r6r*r|rrrrrrdownloadrVr6rrN)rrr*r|rrz*Project-MONAI/model-zoo/hosting_storage_v1ryr0zArepo should be in the form of `repo_owner/repo_name/release_tag`.zEHugging Face Hub repo should be in the form of `repo_owner/repo_name`rrTrzTo download from source: z, `name` must be provided.)rr*rrr)rrrxrr2)rrxr|rr)rrxr|rr)rrevision local_dirz|Currently only download from `url`, source 'github', 'monaihosting', 'huggingface_hub' or 'ngc' are implemented,got source: rl)rFr[rTrrrror rrrjoinrrrospathr0snapshot_downloadNotImplementedError)r*r|rrrrrrr_argsZsource_Z progress_Zremove_prefix_Zrepo_Zname_Zversion_ bundle_dir_Zurl_rrr?r?r@rsxK          rnet_namez1.2z1.5zplease use ``model`` instead.)sinceremoved msg_suffix net_kwargsreturn_state_dict)rrtrainFr?ztorch.nn.Module | Nonez Sequence[str]zstr | BundleWorkflow | Nonez dict | Nonez1object | tuple[torch.nn.Module, dict, dict] | Any)r*rr| workflow_type model_fileload_ts_modulerrrrrdevice key_in_ckpt config_files workflow_namercopy_model_argsr net_overriderrr9c Ks>|r|dk s|dk rtdt|}|dkr2in|}|dkrBin|}| dkr\trXdnd} |dkr~tjd|dkrxdnd}|d krt|}| rt|| d }tj|||}tj |st |||||| | |d |dkrt |t | | d St j|t | d }t|ts2td|dt|dt|}|r<|Sd}|dkr|dkr||d|d}|rdd|D}tf||t||d|}ntd|d|S|dk r t|dstd|S|j}n$|dk r ||d<t|d}|}|| tf|| dkr(|n|| d||S)a Load model weights or TorchScript module of a bundle. Args: name: bundle name. If `None` and `url` is `None`, it must be provided in `args_file`. for example: "spleen_ct_segmentation", "prostate_mri_anatomy" in model-zoo: https://github.com/Project-MONAI/model-zoo/releases/tag/hosting_storage_v1. "monai_brats_mri_segmentation" in ngc: https://catalog.ngc.nvidia.com/models?filters=&orderBy=scoreDESC&query=monai. "mednist_gan" in monaihosting: https://api.ngc.nvidia.com/v2/models/nvidia/monaihosting/mednist_gan/versions/0.2.0/files/mednist_gan_v0.2.0.zip model: a pytorch module to be updated. Default to None, using the "network_def" in the bundle. version: version name of the target bundle to download, like: "0.1.0". If `None`, will download the latest version. If `source` is "huggingface_hub", this argument is a Git revision id. workflow_type: specifies the workflow type: "train" or "training" for a training workflow, or "infer", "inference", "eval", "evaluation" for a inference workflow, other unsupported string will raise a ValueError. default to `train` for training workflow. model_file: the relative path of the model weights or TorchScript module within bundle. If `None`, "models/model.pt" or "models/model.ts" will be used. load_ts_module: a flag to specify if loading the TorchScript module. bundle_dir: directory the weights/TorchScript module will be loaded from. Default is `bundle` subfolder under `torch.hub.get_dir()`. source: storage location name. This argument is used when `model_file` is not existing locally and need to be downloaded first. In default, the value is achieved from the environment variable BUNDLE_DOWNLOAD_SRC, and it should be "ngc", "monaihosting", "github", or "huggingface_hub". repo: repo name. This argument is used when `url` is `None` and `source` is "github" or "huggingface_hub". If `source` is "github", it should be in the form of "repo_owner/repo_name/release_tag". If `source` is "huggingface_hub", it should be in the form of "repo_owner/repo_name". remove_prefix: This argument is used when `source` is "ngc". Currently, all ngc bundles have the ``monai_`` prefix, which is not existing in their model zoo contrasts. In order to maintain the consistency between these three sources, remove prefix is necessary. Therefore, if specified, downloaded folder name will remove the prefix. progress: whether to display a progress bar when downloading. device: target device of returned weights or module, if `None`, prefer to "cuda" if existing. key_in_ckpt: for nested checkpoint like `{"model": XXX, "optimizer": XXX, ...}`, specify the key of model weights. if not nested checkpoint, no need to set. config_files: extra filenames would be loaded. The argument only works when loading a TorchScript module, see `_extra_files` in `torch.jit.load` for more details. workflow_name: specified bundle workflow name, should be a string or class, default to "ConfigWorkflow". args_file: a JSON or YAML file to provide default values for all the args in "download" function. copy_model_args: other arguments for the `monai.networks.copy_model_state` function. return_state_dict: whether to return state dict, if True, return state_dict, else a corresponding network from `_workflow.network_def` will be instantiated and load the achieved weights. net_override: id-value pairs to override the parameters in the network of the bundle, default to `None`. net_name: if not `None`, a corresponding network will be instantiated and load the achieved weights. This argument only works when loading weights. net_kwargs: other arguments that are used to instantiate the network class defined by `net_name`. Returns: 1. If `load_ts_module` is `False` and `model` is `None`, return model weights if can't find "network_def" in the bundle, else return an instantiated network that loaded the weights. 2. If `load_ts_module` is `False` and `model` is not `None`, return an instantiated network that loaded the weights. 3. If `load_ts_module` is `True`, return a triple that include a TorchScript module, the corresponding metadata dict, and extra files dict. please check `monai.data.load_net_with_metadata` for more details. 4. If `return_state_dict` is True, return model weights, only used for compatibility when `model` and `net_name` are all `None`. NzUIncompatible values: model and net_name are all specified, return state dict instead.cuda:0cpumodelsTmodel.tsmodel.ptrr)r*r|rrrrrr) map_locationmore_extra_files)rzthe state dictionary from z should be a dictionary but got rlconfigs.jsoncSsi|]\}}d||qS)z network_def#r?)r=keyvaluer?r?r@ szload..)rr config_filerzCannot find the config file: z, return state dict instead. network_defzINo available network definition in the bundle, return state dict instead._target_)configdstrM)warningswarnrr rrrrrexistsrrtorchrloadr;rrqris_filerGcreate_workflowr<hasattrrr instantiatetor)r*rr|rrrrrrrrrrrrrrrrrrr full_pathrZ _workflowZbundle_config_fileZ _net_overrideZconfigerr?r?r@rsZ            rProject-MONAI/model-zoodevz$dict[str, dict[str, dict[str, Any]]])rrV auth_tokenr9c Csptrb|dkrd|d}nd|d|d}|dk rPdd|i}t||d }nt|}|ntd t|j}td }i}|dkr|D]x} | d |kr| d D]V} | | dd} | |kri|| <| d | dd dd} t | || | <q|Sqn^| D]T} | | d} | |kr:i|| <| | dd} | || dd|| | <q|S)NZhosting_storage_v1zhttps://api.github.com/repos/z /releasesz"https://raw.githubusercontent.com/ryz/models/model_info.json AuthorizationzBearer )headersz0requests package is required, please install it.z_v\d*.rwZassetsr*rrrrr)r*Zbrowser_download_url) rrrrorrrrecompilerreplacer5keys) rrVr request_urlrrZ releases_listZbundle_name_pattern bundles_inforeleaseZassetZ asset_nameZ asset_versionr?r?r@_get_all_bundles_info=s@           rzlist[tuple[str, str]]cCsBt|||d}g}|D]&}t||d}|||fq|S)a* Get all bundles names (and the latest versions) that are stored in the release of specified repository with the provided tag. If tag is "dev", will get model information from https://raw.githubusercontent.com/repo_owner/repo_name/dev/models/model_info.json. The default values of arguments correspond to the release of MONAI model zoo. In order to increase the rate limits of calling Github APIs, you can input your personal access token. Please check the following link for more details about rate limiting: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting The following link shows how to create your personal access token: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token Args: repo: it should be in the form of "repo_owner/repo_name/". tag: the tag name of the release. auth_token: github personal access token. Returns: a list of tuple in the form of (bundle name, latest version). rrVrr)rsortedrrn)rrVrrZ bundles_list bundle_namerr?r?r@get_all_bundles_listis r zdict[str, list[str] | str])r rrVrr9cCsNt|||d}||kr,td|d|d||}t|}|d|dS)a Get the latest version, as well as all existing versions of a bundle that is stored in the release of specified repository with the provided tag. If tag is "dev", will get model information from https://raw.githubusercontent.com/repo_owner/repo_name/dev/models/model_info.json. In order to increase the rate limits of calling Github APIs, you can input your personal access token. Please check the following link for more details about rate limiting: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting The following link shows how to create your personal access token: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token Args: bundle_name: bundle name. repo: it should be in the form of "repo_owner/repo_name/". tag: the tag name of the release. auth_token: github personal access token. Returns: a dictionary that contains the latest version and all versions of a bundle. r bundle: z is not existing in repo: rlr)rrrror r)r rrVrr bundle_inforr?r?r@rs  rzdict[str, Any])r r|rrVrr9cCslt|||d}||kr&td|d||}|dkrFt|d}||krdtd|d|d||S)a Get all information (include "name" and "browser_download_url") of a bundle with the specified bundle name and version which is stored in the release of specified repository with the provided tag. In order to increase the rate limits of calling Github APIs, you can input your personal access token. Please check the following link for more details about rate limiting: https://docs.github.com/en/rest/overview/resources-in-the-rest-api#rate-limiting The following link shows how to create your personal access token: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token Args: bundle_name: bundle name. version: version name of the target bundle, if None, the latest version will be used. repo: it should be in the form of "repo_owner/repo_name/". tag: the tag name of the release. auth_token: github personal access token. Returns: a dictionary that contains the bundle's information. r r z is not existing.Nrz version: z of bundle: r)r r|rrVrrrr?r?r@get_bundle_infosrzstr | Sequence[str] | None) run_idinit_idfinal_id meta_filer logging_filetrackingroverrider9c Ks4tf||||||||d|} | | dS)aF Specify `config_file` to run monai bundle components and workflows. Typical usage examples: .. code-block:: bash # Execute this module as a CLI entry: python -m monai.bundle run --meta_file --config_file # Execute with specified `run_id=training`: python -m monai.bundle run training --meta_file --config_file # Execute with all specified `run_id=runtest`, `init_id=inittest`, `final_id=finaltest`: python -m monai.bundle run --run_id runtest --init_id inittest --final_id finaltest ... # Override config values at runtime by specifying the component id and its new value: python -m monai.bundle run --net#input_chns 1 ... # Override config values with another config file `/path/to/another.json`: python -m monai.bundle run --net %/path/to/another.json ... # Override config values with part content of another config file: python -m monai.bundle run --net %/data/other.json#net_arg ... # Set default args of `run` in a JSON / YAML file, help to record and simplify the command line. # Other args still can override the default args at runtime: python -m monai.bundle run --args_file "/workspace/data/args.json" --config_file Args: run_id: ID name of the expected config expression to run, default to "run". to run the config, the target config must contain this ID. init_id: ID name of the expected config expression to initialize before running, default to "initialize". it's optional for both configs and this `run` function. final_id: ID name of the expected config expression to finalize after running, default to "finalize". it's optional for both configs and this `run` function. meta_file: filepath of the metadata file, if it is a list of file paths, the content of them will be merged. Default to None. config_file: filepath of the config file, if `None`, must be provided in `args_file`. if it is a list of file paths, the content of them will be merged. logging_file: config file for `logging` module in the program. for more details: https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig. Default to None. tracking: if not None, enable the experiment tracking at runtime with optionally configurable and extensible. If "mlflow", will add `MLFlowHandler` to the parsed bundle with default tracking settings where a set of common parameters shown below will be added and can be passed through the `override` parameter of this method. - ``"output_dir"``: the path to save mlflow tracking outputs locally, default to "/eval". - ``"tracking_uri"``: uri to save mlflow tracking outputs, default to "/output_dir/mlruns". - ``"experiment_name"``: experiment name for this run, default to "monai_experiment". - ``"run_name"``: the name of current run. - ``"save_execute_config"``: whether to save the executed config files. It can be `False`, `/path/to/artifacts` or `True`. If set to `True`, will save to the default path "/eval". Default to `True`. If other string, treat it as file path to load the tracking settings. If `dict`, treat it as tracking settings. Will patch the target config content with `tracking handlers` and the top-level items of `configs`. for detailed usage examples, please check the tutorial: https://github.com/Project-MONAI/tutorials/blob/main/experiment_management/bundle_integrate_mlflow.ipynb. args_file: a JSON or YAML file to provide default values for `run_id`, `meta_file`, `config_file`, `logging`, and override pairs. so that the command line inputs can be simplified. override: id-value pairs to override or add the corresponding config content. e.g. ``--net#input_chns 42``, ``--net %/data/other.json#net_arg``. )rrrrrrrrNrrunfinalize) rrrrrrrrrZworkflowr?r?r@rsM  r)rrr8r9cKs(tf||d|}||dS)a Specify `bundle workflow` to run monai bundle components and workflows. The workflow should be subclass of `BundleWorkflow` and be available to import. It can be MONAI existing bundle workflows or user customized workflows. Typical usage examples: .. code-block:: bash # Execute this module as a CLI entry with default ConfigWorkflow: python -m monai.bundle run_workflow --meta_file --config_file # Set the workflow to other customized BundleWorkflow subclass: python -m monai.bundle run_workflow --workflow_name CustomizedWorkflow ... Args: workflow_name: specified bundle workflow name, should be a string or class, default to "ConfigWorkflow". args_file: a JSON or YAML file to provide default values for this API. so that the command line inputs can be simplified. kwargs: arguments to instantiate the workflow class. )rrNr)rrr8 workflow_r?r?r@ run_workflow2srz bool | None)rr create_dirr hash_typerr8r9c Kstf||||||d|}td|dt|dddddd \}} } } } t|| d tj| d } | d }|dkrxtd t||| | ddtj |d}zt f| |d|WnPt k r}z2tt dt jt|dd|d|W5d}~XYnXtddS)aV Verify the provided `metadata` file based on the predefined `schema`. `metadata` content must contain the `schema` field for the URL of schema file to download. The schema standard follows: http://json-schema.org/. Args: meta_file: filepath of the metadata file to verify, if `None`, must be provided in `args_file`. if it is a list of file paths, the content of them will be merged. filepath: file path to store the downloaded schema. create_dir: whether to create directories if not existing, default to `True`. hash_val: if not None, define the hash value to verify the downloaded schema file. hash_type: if not None, define the hash type to verify the downloaded schema file. Defaults to "md5". args_file: a JSON or YAML file to provide default values for all the args in this function. so that the command line inputs can be simplified. kwargs: other arguments for `jsonschema.validate()`. for more details: https://python-jsonschema.readthedocs.io/en/stable/validate/#jsonschema.validate. )r6rrrrrverify_metadatarrrTNmd5)rrr)rr)filesschemazKmust provide the `schema` field in the metadata for the URL of schema file.)rrrrr)r)instancer"z.*Failed validatingrz against schema ``.z#metadata is verified with no error.)rFr[rTr!rload_config_filesr.rorrBr(r+rrSfindallr<rWrX)rrrrrrr8r filepath_ meta_file_Z create_dir_Z hash_val_Z hash_type_metadatarr"er?r?r@rQsF     &r_meta_#network_data_formatzConfigParser | None)parserrr9c Cst|tstdt|d|d}|d}||}|d}t||}|d}t||tj}|d}|d}||}|d }t||tj}|||||fS) a Get the input and output information defined in the metadata. Args: parser: a ConfigParser of the given bundle. prefix: a prefix for the input and output ID, which will be combined as `prefix#inputs` and `prefix#outputs` to parse the input and output information in the `metadata.json` file of a bundle, default to `meta_#network_data_format`. Returns: input_channels: the channel number of the `image` input. input_spatial_shape: the spatial shape of the `image` input. input_dtype: the data type of the `image` input. output_channels: the channel number of the output. output_dtype: the data type of the output. z/Parameter parser should be a ConfigParser, got rlz#inputsz#image#num_channelsz#image#spatial_shapez #image#dtypez#outputsz#pred#num_channelsz #pred#dtype) r;rAttributeErrorrqr.rCr$rTensor) r-rZ prefix_keyrinput_channelsinput_spatial_shape input_dtypeoutput_channels output_dtyper?r?r@_get_net_io_infos          r5r)r-r9cCs,t|d\}}}}}t|}d|f|}|S)z Get a fake input shape e.g. [N, C, H, W] or [N, C, H, W, D], whose batch size is 1, from the given parser. Args: parser: a ConfigParser which contains the i/o information of a bundle. r-re)r5rt)r-r0r1_ spatial_shape input_shaper?r?r@_get_fake_input_shapes r:z int | None) net_idrrrrhrirjextra_forward_argsrrr9c " Kstf|||||||||d | } td| dt| dddtrBdndd d d id \} } } }}}}}t}|j| d |j| d | D]\}}|||<qt|d \}}}}}z| }| | |}Wn4t k r}zt d |d|W5d}~XYnX| t t||||d}t jd |f|||d}|t jkr| dddlm} | ||f|}!W5QRX| |n ||f|}!|!jd |krtd|!jd d|d|!j|krtd|!jd|dW5QRXtddS)a Verify the input and output data shape and data type of network defined in the metadata. Will test with fake Tensor data according to the required data shape in `metadata`. Typical usage examples: .. code-block:: bash python -m monai.bundle verify_net_in_out network --meta_file --config_file Args: net_id: ID name of the network component to verify, it must be `torch.nn.Module`. meta_file: filepath of the metadata file to get network args, if `None`, must be provided in `args_file`. if it is a list of file paths, the content of them will be merged. config_file: filepath of the config file to get network definition, if `None`, must be provided in `args_file`. if it is a list of file paths, the content of them will be merged. device: target device to run the network forward computation, if None, prefer to "cuda" if existing. p: power factor to generate fake data shape if dim of expected shape is "x**p", default to 1. n: multiply factor to generate fake data shape if dim of expected shape is "x*n", default to 1. any: specified size to generate fake data shape if dim of expected shape is "*", default to 1. extra_forward_args: a dictionary that contains other args for the forward function of the network. Default to an empty dictionary. args_file: a JSON or YAML file to provide default values for `net_id`, `meta_file`, `config_file`, `device`, `p`, `n`, `any`, and override pairs. so that the command line inputs can be simplified. override: id-value pairs to override or add the corresponding config content. e.g. ``--_meta#network_data_format#inputs#image#num_channels 3``. ) r6r;rrrrhrirjr<verify_net_in_outrrrrrrre)r;rrhrirjr<fr6z.F_meta_N)jit_objfilename_prefix_or_streaminclude_config_valsappend_timestamp meta_valuesrzexported to file: rl)rE has_igniter, load_objectsrrrr#rrbasenamesplitextrordumpsrrBencoderGrr.rOrWrX)rRr-r;rrSrrr8rOckpt extra_filesrPrxr7r?r?r@_export&s.    rdzSequence[int] | NonezMapping | None) r;rrSrrr use_tracer9rconverter_kwargsrr9c  Ks2tf|||||||||| d | } td| dt| ddddddd did \ } } }}}}}}}t}|j|d |dk r|j|d | D]\}}|||<q|st|d }t |g}| |}t rt j ||i| d n&t| }t||dkr|n||d|||dtfd|i|}t|| dS)a Export the model checkpoint to an onnx model. Typical usage examples: .. code-block:: bash python -m monai.bundle onnx_export network --filepath --ckpt_file ... Args: net_id: ID name of the network component in the config, it must be `torch.nn.Module`. filepath: filepath where the onnx model is saved to. ckpt_file: filepath of the model checkpoint to load. meta_file: filepath of the metadata file, if it is a list of file paths, the content of them will be merged. config_file: filepath of the config file that contains extract network information, key_in_ckpt: for nested checkpoint like `{"model": XXX, "optimizer": XXX, ...}`, specify the key of model weights. if not nested checkpoint, no need to set. use_trace: whether using `torch.jit.trace` to convert the pytorch model to torchscript model. input_shape: a shape used to generate the random input of the network, when converting the model to an onnx model. Should be a list like [N, C, H, W] or [N, C, H, W, D]. If not given, will try to parse from the `metadata` config. args_file: a JSON or YAML file to provide default values for all the parameters of this function, so that the command line inputs can be simplified. converter_kwargs: extra arguments that are needed by `convert_to_onnx`, except ones that already exist in the input parameters. override: id-value pairs to override or add the corresponding config content. e.g. ``--_meta#network_data_format#inputs#image#num_channels 3``. r6r;rrrrSrrer9rf onnx_exportrrrSrrNF)r;rrrer9rfr>r6rTrinputsrer)rFr[rTrrCrDrGr:rrHrEr\r,r]rrupdaterr/save)r;rrSrrrrer9rrfrrr( ckpt_file_rJrKr) key_in_ckpt_ use_trace_ input_shape_converter_kwargs_r-rKrLinputs_rOrb onnx_modelr?r?r@rhhsl*         rhc  Kstf|||||||||| d | } td| dt| ddddddddid \ } } }}}}}}}| d t}t}|j| d |dkrtj |d d n|}tj |r|j |d | D]\}}|||<q| dkrtj |d dn| } |dkrtj |d dn|}tj |s&t d|d|dkr4dn|}z||Wn<tk r}ztd|d| d|W5d}~XYnX|s|rt|d}|rt|gnd}|||dtt|f|| || |d|dS)a Export the model checkpoint to the given filepath with metadata and config included as JSON files. Typical usage examples: .. code-block:: bash python -m monai.bundle ckpt_export network --filepath --ckpt_file ... Args: net_id: ID name of the network component in the config, it must be `torch.nn.Module`. Default to "network_def". filepath: filepath to export, if filename has no extension it becomes `.ts`. Default to "models/model.ts" under "os.getcwd()" if `bundle_root` is not specified. ckpt_file: filepath of the model checkpoint to load. Default to "models/model.pt" under "os.getcwd()" if `bundle_root` is not specified. meta_file: filepath of the metadata file, if it is a list of file paths, the content of them will be merged. Default to "configs/metadata.json" under "os.getcwd()" if `bundle_root` is not specified. config_file: filepath of the config file to save in TorchScript model and extract network information, the saved key in the TorchScript model is the config filename without extension, and the saved config value is always serialized in JSON format no matter the original file format is JSON or YAML. it can be a single file or a list of files. if `None`, must be provided in `args_file`. key_in_ckpt: for nested checkpoint like `{"model": XXX, "optimizer": XXX, ...}`, specify the key of model weights. if not nested checkpoint, no need to set. use_trace: whether using `torch.jit.trace` to convert the PyTorch model to TorchScript model. input_shape: a shape used to generate the random input of the network, when converting the model to a TorchScript model. Should be a list like [N, C, H, W] or [N, C, H, W, D]. If not given, will try to parse from the `metadata` config. args_file: a JSON or YAML file to provide default values for all the parameters of this function, so that the command line inputs can be simplified. converter_kwargs: extra arguments that are needed by `convert_to_torchscript`, except ones that already exist in the input parameters. override: id-value pairs to override or add the corresponding config content. e.g. ``--_meta#network_data_format#inputs#image#num_channels 3``. rg ckpt_exportrrNrF)rrSr;rrrer9rf bundle_rootr>r metadata.jsonrrrzCheckpoint file "z7" not found, please specify it in argument "ckpt_file".rzNetwork definition "z" cannot be found in "z'", specify name with argument "net_id".r6rir;rrSrr)rFr[rTr.rgetcwdrrCrrrrDrGFileNotFoundErrorrEror:rrHrkrdr)r;rrSrrrrer9rrfrrrJr(rmrKr)rnrorprqrur-rKrLr+rrr?r?r@rts1         rtzSequence[str] | None)r;rrSrrr precisionr9redynamic_batchsizeruse_onnxonnx_input_namesonnx_output_namesrrfrr9c%Kstf||||||||||| | | | | |d|}td|dt|ddddddd gd ddd d gd gid \}}}}}}}}}}}}}}} t}!|!j|d|dk r|!j|d|D]\}"}#|#|!|"<q|st|!d}||||||||d}$| |$t t |!f|||||d| dS)a Export the model checkpoint to the given filepath as a TensorRT engine-based TorchScript. Currently, this API only supports converting models whose inputs are all tensors. There are two ways to export a model: 1, Torch-TensorRT way: PyTorch module ---> TorchScript module ---> TensorRT engine-based TorchScript. 2, ONNX-TensorRT way: PyTorch module ---> TorchScript module ---> ONNX model ---> TensorRT engine ---> TensorRT engine-based TorchScript. When exporting through the first way, some models suffer from the slowdown problem, since Torch-TensorRT may only convert a little part of the PyTorch model to the TensorRT engine. However when exporting through the second way, some Python data structures like `dict` are not supported. And some TorchScript models are not supported by the ONNX if exported through `torch.jit.script`. Typical usage examples: .. code-block:: bash python -m monai.bundle trt_export --net_id --filepath --ckpt_file --input_shape --dynamic_batchsize ... Args: net_id: ID name of the network component in the config, it must be `torch.nn.Module`. filepath: filepath to export, if filename has no extension, it becomes `.ts`. ckpt_file: filepath of the model checkpoint to load. meta_file: filepath of the metadata file, if it is a list of file paths, the content of them will be merged. config_file: filepath of the config file to save in the TensorRT based TorchScript model and extract network information, the saved key in the model is the config filename without extension, and the saved config value is always serialized in JSON format no matter the original file format is JSON or YAML. it can be a single file or a list of files. if `None`, must be provided in `args_file`. key_in_ckpt: for nested checkpoint like `{"model": XXX, "optimizer": XXX, ...}`, specify the key of model weights. if not nested checkpoint, no need to set. precision: the weight precision of the converted TensorRT engine based TorchScript models. Should be 'fp32' or 'fp16'. input_shape: the input shape that is used to convert the model. Should be a list like [N, C, H, W] or [N, C, H, W, D]. If not given, will try to parse from the `metadata` config. use_trace: whether using `torch.jit.trace` to convert the PyTorch model to a TorchScript model and then convert to a TensorRT engine based TorchScript model or an ONNX model (if `use_onnx` is True). dynamic_batchsize: a sequence with three elements to define the batch size range of the input for the model to be converted. Should be a sequence like [MIN_BATCH, OPT_BATCH, MAX_BATCH]. After converted, the batchsize of model input should between `MIN_BATCH` and `MAX_BATCH` and the `OPT_BATCH` is the best performance batchsize that the TensorRT tries to fit. The `OPT_BATCH` should be the most frequently used input batchsize in the application. device: the target GPU index to convert and verify the model. use_onnx: whether using the ONNX-TensorRT way to export the TensorRT engine-based TorchScript model. onnx_input_names: optional input names of the ONNX model. This arg is only useful when `use_onnx` is True. Should be a sequence like `['input_0', 'input_1', ..., 'input_N']` where N equals to the number of the model inputs. If not given, will use `['input_0']`, which supposes the model only has one input. onnx_output_names: optional output names of the ONNX model. This arg is only useful when `use_onnx` is True. Should be a sequence like `['output_0', 'output_1', ..., 'output_N']` where N equals to the number of the model outputs. If not given, will use `['output_0']`, which supposes the model only has one output. args_file: a JSON or YAML file to provide default values for all the parameters of this function, so that the command line inputs can be simplified. converter_kwargs: extra arguments that are needed by `convert_to_trt`, except ones that already exist in the input parameters. override: id-value pairs to override or add the corresponding config content. e.g. ``--_meta#network_data_format#inputs#image#num_channels 3``. )r6r;rrrrSrrzr9rer{rr|r}r~rf trt_exportrrrSrrNfp32Finput_0output_0) r;rrrzr9rer{rr|r}r~rfr>r6)rzr9r{rerr|r}r~rw) rFr[rTrrCrDrGr:rkrdr)%r;rrSrrrrzr9rer{rr|r}r~rrfrrr(rmrJrKr)rnZ precision_rproZdynamic_batchsize_rLZ use_onnx_Zonnx_input_names_Zonnx_output_names_rqr-rKrLZtrt_api_parametersr?r?r@rVsM      rrzdict | str | None)rrSnetworkdataset_license metadata_str inference_strr9c Cs|dkr t}|dkrt}t|}|r@tdt|d|js^tdt|d|d}|d}|d}| | | | t |t rt j |d d }t |t rt j |d d }tt|d d } | |W5QRXtt|d d } | |W5QRXtt|dd } d} | t| W5QRXtt|dd } | dW5QRX|dkrtt|dd } | dW5QRX|dk rtt|t|dn|dk rt|t|ddS)a Initialise a new bundle directory with some default configuration files and optionally network weights. Typical usage example: .. code-block:: bash python -m monai.bundle init_bundle /path/to/bundle_dir network_ckpt.pt Args: bundle_dir: directory name to create, must not exist but parent direct must exist ckpt_file: optional checkpoint file to copy into bundle network: if given instead of ckpt_file this network's weights will be stored in bundle dataset_license: if `True`, a default license file called "data_license.txt" will be produced. This file is required if there are any license conditions stated for data your bundle uses. metadata_str: optional metadata string to write to bundle, if not given a default will be used. inference_str: optional inference string to write to bundle, if not given a default will be used. NzSpecified bundle directory 'z' already existsz0Parent directory of specified bundle directory 'z' does not existrrdocs)indentrvwzinference.json README.mda # Your Model Name Describe your model here and how to run it, for example using `inference.json`: ``` python -m monai.bundle run --meta_file /path/to/bundle/configs/metadata.json --config_file /path/to/bundle/configs/inference.json --dataset_dir ./input --bundle_root /path/to/bundle ``` LICENSEz*Select a license and place its terms here Tzdata_license.txtz6Select a license for dataset and place its terms here r)rrrabsoluterror<parentis_dirmkdirr;r5rr`openwriterrr ) rrSrrrrZ configs_dirZ models_dirZdocs_diroreadmer?r?r@ init_bundlesH       rc Csd}tjtj|d}tj|rdt|}|}W5QRXd|krXd|krXd}n d|krdd}d}d |d |d }t|}|}W5QRX|d |}t|d }||W5QRXdS)NunknownrzApache Licensez Version 2.0z apache-2.0z MIT LicenseZmitz&- monai - medical library_name: monai z --- tags: z license: z --- r)rrrdirnamerrreadr)new_modelcard_pathZ license_nameZ license_pathfilecontentrZ tag_contentZ new_contentr?r?r@_add_model_card_metadata[s      r) rr*rtokenprivater|tag_as_latest_versionupload_folder_kwargsr9c Kstj|d}|j||ddtj||d} tj||dd} tj| r\t| | t| |j f|tj||d|} |dk r|j ||dd|r|j |d dd| S) af Push a MONAI bundle to the Hugging Face Hub. Typical usage examples: .. code-block:: bash python -m monai.bundle push_to_hf_hub --repo --name --bundle_dir --version ... Args: repo: namespace (user or organization) and a repo name separated by a /, e.g. `hf_username/bundle_name` bundle_name: name of the bundle directory to push. bundle_dir: path to the bundle directory. token: Hugging Face authentication token. Default is `None` (will default to the stored token). private: Private visibility of the repository on Hugging Face. Default is `True`. version_name: Name of the version tag to create. Default is `None` (no version tag is created). tag_as_latest_version: Whether to tag the commit as `latest_version`. This version will downloaded by default when using `bundle.download()`. Default is `False`. upload_folder_kwargs: Keyword arguments to pass to `HfApi.upload_folder`. Returns: repo_url: URL of the Hugging Face repo )rT)rrexist_okrr)r folder_pathN)rrVrr) r0HfApi create_reporrrrrr upload_folder create_tag) rr*rrrr|rrhf_apirZmodelcard_pathZrepo_urlr?r?r@push_to_hf_hubss#    r)rrrr8r9cKstf|||d|}td|dt|tdd\}}t|tr|tdt|d\}}|sbtt|}|dkrtd|d n t |t r|}ntd |d |dk r|fd |i|}n |f|}| |S) a2 Specify `bundle workflow` to create monai bundle workflows. The workflow should be subclass of `BundleWorkflow` and be available to import. It can be MONAI existing bundle workflows or user customized workflows. Typical usage examples: .. code-block:: python # Specify config_file path to create workflow: workflow = create_workflow(config_file="/workspace/spleen_ct_segmentation/configs/train.json", workflow_type="train") # Set the workflow to other customized BundleWorkflow subclass to create workflow: workflow = create_workflow(workflow_name=CustomizedWorkflow) Args: workflow_name: specified bundle workflow name, should be a string or class, default to "ConfigWorkflow". config_file: filepath of the config file, if it is a list of file paths, the content of them will be merged. args_file: a JSON or YAML file to provide default values for this API. so that the command line inputs can be simplified. kwargs: arguments to instantiate the workflow class. )r6rrrrN)rrz monai.bundler)z(cannot locate specified workflow class: rlzaArgument `workflow_name` must be a bundle workflow class nameor subclass of BundleWorkflow, got: r) rFr[rTrr;r<r&rro issubclassr initialize)rrrr8rZworkflow_class has_built_inrr?r?r@rs.       r) bundle_pathlarge_file_namer9cCs|dkrtn|}|dkr\tt|d}ttdd|}t|dkr\td|dt}| || d}|D]z}d |d <d |kr| d d d kr| d d |kr| d d d kr| d tj ||d|d<| dtf|q|dS)a This utility allows you to download large files from a bundle. It supports file suffixes like ".yml", ".yaml", and ".json". If you don't specify a `large_file_name`, it will automatically search for large files among the supported suffixes. Typical usage examples: .. code-block:: bash # Execute this module as a CLI entry to download large files from a bundle path: python -m monai.bundle download_large_files --bundle_path # Execute this module as a CLI entry to download large files from the bundle path with a specified `large_file_name`: python -m monai.bundle download_large_files --bundle_path --large_file_name large_files.yaml Args: bundle_path: (Optional) The path to the bundle where the files are located. Default is `os.getcwd()`. large_file_name: (Optional) The name of the large file to be downloaded. Nz large_files*cSs |jdkS)N)z.ymlz.yamlr)suffix)r>r?r?r@z&download_large_files..rz0Cannot find the large_files.yml/yaml/json under rlZ large_filesTfuzzyrrrrr)rrxrDrglobfilterrryrrCr.rOrrr)rrZlarge_file_pathr-Zlarge_files_listZlf_datar?r?r@download_large_filess$      r)NT)rerere)T)r)r)N)rrN)rrN)rrN)NrrN)NNNNNNNN)NN)NNNNNN)Nr,) NNNNNNNNN) NNNNNNNNNN) NNNNNNNNNN)NNNNNNNNNNNNNNNN)NNFNN)NTNF)NNN)NN)t __future__rr]rrrrcollections.abcrrpathlibrpydocrshutilrtextwraprtypingr r r torch.cudar Zmonai.apps.mmars.mmarsr monai.apps.utilsr rrrZmonai.bundle.config_itemrmonai.bundle.config_parserrmonai.bundle.utilsrrZmonai.bundle.workflowsrr monai.configrr monai.datarrmonai.networksrrrrrr monai.utilsr!r"r#r$r%r&r'r(r7r+OPT_IMPORT_VERSIONr,r\rrr/r0__name__rWenvironr.ZDEFAULT_DOWNLOAD_SOURCErYrF _update_argsrTr[rdrtrzrrrrrrrrrrrrrr rrrrrr5r:r=rdrhrtrrrrrrr?r?r?r@ s~          $    %    " @%-#$*"] =%$fC&m&2.Z =: