o % i@s"ddlmZddlZddlZddlZddlZddlmZmZddl m Z ddl m Z ddl m Z ddlmZddlmZdd lmZdd lmZdd lmZmZmZdd lmZmZmZmZdd lm Z ddl!m"Z"m#Z#m$Z$ddgZ%ee&dZ'GdddeZ(Gddde(Z)Gddde(Z*dS)) annotationsN)ABCabstractmethod)Sequence)copy) fileConfig)Path)Any) get_logger) ConfigParser)InferPropertiesMetaPropertiesTrainProperties)DEFAULT_EXP_MGMT_SETTINGSEXPR_KEY ID_REF_KEY ID_SEP_KEY)PathLike)BundlePropertyBundlePropertyConfig ensure_tupleBundleWorkflowConfigWorkflow) module_namecseZdZUdZdZded<dZded<    d4d5ddZed6ddZ ed6ddZ ed6ddZ ed7dd Z ed8d"d#Z d$d%Zfd&d'Zd(d)Zd*d+Zd9d:d/d0Zd;d2d3ZZS)No 'meta' key found in properties while workflow_type is None.z not found in property file z'Error decoding JSON from property file zKNo workflow type and property file specified, default to 'meta' properties.)ospathisfileFileNotFoundErrorloggerinfor isinstancestrerrorlistlowerrr# ValueErrorris_fileopenjsonload propertiesupdatewarningKeyErrorJSONDecodeErrorrr r rr$r() selfr$r&r(r*f json_filer@erIX/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/bundle/workflows.py__init__@s                  zBundleWorkflow.__init__argsr kwargsreturncOt)A Initialize the bundle workflow before running. NotImplementedErrorrErLrMrIrIrJ initializezBundleWorkflow.initializecOrO)zZ Run the bundle workflow, it can be a training, evaluation or inference. rQrSrIrIrJrunrUzBundleWorkflow.runcOrO)F Finalize step after the running of bundle workflow. rQrSrIrIrJfinalizerUzBundleWorkflow.finalizenamer7propertydictcCrO)a With specified property name and information, get the expected property value. Args: name: the name of target property. property: other information for the target property, defined in `TrainProperties` or `InferProperties`. rQ)rErYrZrIrIrJ _get_propertys zBundleWorkflow._get_propertyvaluecCrO)I With specified property name and information, set value for the expected property. Args: name: the name of target property. property: other information for the target property, defined in `TrainProperties` or `InferProperties`. value: value to set for the property. rQrErYrZr]rIrIrJ _set_propertys zBundleWorkflow._set_propertycCs2|jdur||jvr|j||j|dS||S)NrYrZ)r@r\__getattribute__)rErYrIrIrJ __getattr__s zBundleWorkflow.__getattr__csH|dkr|jdur||jvr|j||j||ddSt||dS)Nr@)rYrZr])r@r`super __setattr__)rErYr] __class__rIrJreszBundleWorkflow.__setattr__cC|jS)zP Get the workflow type, it can be `None`, "train", or "infer". )r$rErIrIrJget_workflow_typez BundleWorkflow.get_workflow_typecCrh)z% Get the meta file. )r(rirIrIrJ get_meta_filerkzBundleWorkflow.get_meta_filerequireddescNonecCsF|jduri|_||jvrtd|dtj|tj|i|j|<dS)a Besides the default predefined properties, some 3rd party applications may need the bundle definition to provide additional properties for the specific use cases, if the bundle can't provide the property, means it can't work with the application. This utility adds the property for the application requirements check and access. Args: name: the name of target property. required: whether the property is "must-have". desc: descriptions for the property. Nz property 'z7' already exists in the properties list, overriding it.)r@r4rBrDESCREQUIRED)rErYrmrnrIrIrJ add_propertys  zBundleWorkflow.add_propertylist[str] | Nonecs&jdurdSfddjDS)z Check whether the required properties are existing in the bundle workflow. If no workflow type specified, return None, otherwise, return a list of required but missing properties. Ncs,g|]\}}|tjdrt|s|qS)F)getrrqhasattr).0nprirIrJ s,z3BundleWorkflow.check_properties..)r@itemsrirIrirJcheck_propertiess zBundleWorkflow.check_properties)NNNN)r$r%r&r'r(r)r*r%rLr rMr rNr rYr7rZr[rNr rYr7rZr[r]r rNr N)rYr7rmr7rnr%rNrorNrs)__name__ __module__ __qualname____doc__r__annotations__r#rKrrTrVrXr\r`rcrerjrlrrr{ __classcell__rIrIrfrJr%s2    K       csfeZdZUdZdZded<dZded<     d"d#fdd Zd$ddZd%ddZ d&d d!Z Z S)'PythonicWorkflowai Base class for the pythonic workflow specification in bundle, it can be a training, evaluation or inference workflow. It defines the basic interfaces for the bundle workflow behavior: `initialize`, `finalize`, etc. This also provides the interface to get / set public properties to interact with a bundle workflow through defined `get_` accessor methods or directly defining members of the object. For how to set the properties, users can define the `_set_` methods or directly set the members of the object. The `initialize` method is called to set up the workflow before running. This method sets up internal state and prepares properties. If properties are modified after the workflow has been initialized, `self._is_initialized` is set to `False`. Before running the workflow again, `initialize` should be called to ensure that the workflow is properly set up with the new property values. Args: workflow_type: specifies the workflow type: "train" or "training" for a training workflow, or "infer", "inference", "eval", "evaluation" for a inference workflow, other unsupported string will raise a ValueError. default to `None` for only using meta properties. workflow: specifies the workflow type: "train" or "training" for a training workflow, or "infer", "inference", "eval", "evaluation" for a inference workflow, other unsupported string will raise a ValueError. default to `None` for common workflow. properties_path: the path to the JSON file of properties. If `workflow_type` is specified, properties will be loaded from the file based on the provided `workflow_type` and meta. If no `workflow_type` is specified, properties will default to loading from "meta". If `properties_path` is None, default properties will be sourced from "monai/bundle/properties.py" based on the workflow_type: For a training workflow, properties load from `TrainProperties` and `MetaProperties`. For a inference workflow, properties load from `InferProperties` and `MetaProperties`. For workflow_type = None : only `MetaProperties` will be loaded. config_file: path to the config file, typically used to store hyperparameters. meta_file: filepath of the metadata file, if this is a list of file paths, their contents will be merged in order. logging_file: config file for `logging` module in the program. for more details: https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig. rrrrr#Nr$r%r&r' config_filer)r(r*overrider c s|durtttdn|}tj||||di|_i|_t|_ |dur/|j j |d|j dur<|j j |j d|j j |dd|_dS)N metadata.json)r$r&r(r*rFpairsF)r7rr0getcwdrdrK _props_vals_set_props_valsr parser read_configr( read_metarA_is_initialized)rEr$r&rr(r*rrfrIrJrKs   zPythonicWorkflow.__init__rLrMrNcOsi|_d|_dS)z@ Initialize the bundle workflow before running. TN)rrrSrIrIrJrT2s zPythonicWorkflow.initializerYr7rZr[c Cs|jstdd}||jvr|j|}|S||jvr!|j|}|S||jj|jjvr>|j|dt j d}|j|}|Sz t |d|}Wn#t yl}z|t jrbtd|d|d|WYd}~nd}~ww||j|<|S)a With specified property name and information, get the expected property value. If the property is already generated, return from the bucket directly. If user explicitly set the property, return it directly. Otherwise, generate the expected property as a class private property with prefix "_". Args: name: the name of target property. property: other information for the target property, defined in `TrainProperties` or `InferProperties`. z:Please execute 'initialize' before getting any properties.Nget_zunsupported property 'zG' is required in the bundle properties,need to implement a method 'get_z' to provide the property.)r RuntimeErrorrrrconfigmeta_keyr@rtrIDgetattrAttributeErrorrrqr;)rErYrZr]idrHrIrIrJr\9s:         zPythonicWorkflow._get_propertyr]cCs||j|<d|_dS)a With specified property name and information, set value for the expected property. Stores user-reset initialized objects that should not be re-initialized and marks the workflow as not initialized. Args: name: the name of target property. property: other information for the target property, defined in `TrainProperties` or `InferProperties`. value: value to set for the property. FN)rrr_rIrIrJr`Zs zPythonicWorkflow._set_property)NNNNN) r$r%r&r'rr)r(r)r*r%rr r|r}r~) rrrrrrr#rKrTr\r`rrIrIrfrJrs  "   !rcseZdZdZ        dBdCfdd ZdDddZdDdd ZdDd!d"ZdEfd$d% ZdFd*d+Z dGd.d/Z dGd0d1Z dHd3d4Z dIdJfd8d9 Z dKd;d<ZedLd@dAZZS)Mrad Specification for the config-based bundle workflow. Standardized the `initialize`, `run`, `finalize` behavior in a config-based training, evaluation, or inference. Before `run`, we add bundle root directory to Python search directories automatically. For more information: https://docs.monai.io/en/latest/mb_specification.html. Args: config_file: filepath of the config file, if this is a list of file paths, their contents will be merged in order. meta_file: filepath of the metadata file, if this is a list of file paths, their contents will be merged in order. If None, default to "configs/metadata.json", which is commonly used for bundles in MONAI model zoo. logging_file: config file for `logging` module in the program. for more details: https://docs.python.org/3/library/logging.config.html#logging.config.fileConfig. If None, default to "configs/logging.conf", which is commonly used for bundles in MONAI model zoo. If False, the logging logic for the bundle will not be modified. init_id: ID name of the expected config expression to initialize before running, default to "initialize". allow a config to have no `initialize` logic and the ID. run_id: ID name of the expected config expression to run, default to "run". to run the config, the target config must contain this ID. final_id: ID name of the expected config expression to finalize after running, default to "finalize". allow a config to have no `finalize` logic and the ID. tracking: if not None, enable the experiment tracking at runtime with optionally configurable and extensible. if "mlflow", will add `MLFlowHandler` to the parsed bundle with default tracking settings, if other string, treat it as file path to load the tracking settings. if `dict`, treat it as tracking settings. will patch the target config content with `tracking handlers` and the top-level items of `configs`. for detailed usage examples, please check the tutorial: https://github.com/Project-MONAI/tutorials/blob/main/experiment_management/bundle_integrate_mlflow.ipynb. workflow_type: specifies the workflow type: "train" or "training" for a training workflow, or "infer", "inference", "eval", "evaluation" for a inference workflow, other unsupported string will raise a ValueError. default to `None` for common workflow. properties_path: the path to the JSON file of properties. If `workflow_type` is specified, properties will be loaded from the file based on the provided `workflow_type` and meta. If no `workflow_type` is specified, properties will default to loading from "train". If `properties_path` is None, default properties will be sourced from "monai/bundle/properties.py" based on the workflow_type: For a training workflow, properties load from `TrainProperties` and `MetaProperties`. For a inference workflow, properties load from `InferProperties` and `MetaProperties`. For workflow_type = None : only `MetaProperties` will be loaded. override: id-value pairs to override or add the corresponding config content. e.g. ``--net#input_chns 42``, ``--net %/data/other.json#net_arg`` NrTrVrXrrstr | Sequence[str]r(r)r*str | bool | Noneinit_idr7run_idfinal_idtrackingstr | dict | Noner$r%r&r'rr rNroc  s|dur6t|} t| dj} | D]#} t| } | j| kr(td| d| d| s4td| dqntd} |durDt| dn|}tj ||| d | |_ |dur]t|j d n|}|d urmtd |d n2t j |s|t|j d krtd|dntd|dtt|d dtd|dt|_|jj|d|jdur|jj|jd|jj| d||_||_||_|durt|tr|tvrt|}nt|}|j|j|dd |_dS)NrzNot all config files are in z2. If logging_file and meta_file arenot specified, z3 will be used as the default config root directory.zCannot find the config file: r,configsr)r$r(r&z logging.confFzLogging file is set to z, skipping logging.zDefault logging file in z" does not exist, skipping logging.r+r.r-rr)rsettings)rrparentr4rBr<r3r7rdrKconfig_root_pathr0r1r2rr5r rrr(rrArrrr6rload_config_filespatch_bundle_trackingr)rErr(r*rrrrr$r&rZ _config_filesrZ _config_fileZ settings_rfrIrJrKsT        zConfigWorkflow.__init__cCs"|jjddd|_|j|jdS)rPT)resetr)rparser _run_exprrrirIrIrJrTszConfigWorkflow.initializecCsX|jjdkr |jjn|j}tjdt||j|jvr%t d|jd|j |jdS)z Run the bundle workflow, it can be a training, evaluation or inference. Before run, we add bundle root directory to Python search directories automatically. rzrun ID 'z#' doesn't exist in the config file.r) rrYrsysr1insertr7rrr;r)rEZ_bundle_root_pathrIrIrJrVs  zConfigWorkflow.runcCs|j|jdS)rWr)rrrirIrIrJrXszConfigWorkflow.finalizerscst}|jdurtddS|rtd|g}|jD]\}}|tjds9|j ||ds9| |q"|rDtd||durM| ||S)aD Check whether the required properties are existing in the bundle workflow. If the optional properties have reference in the config, will also check whether the properties are existing. If no workflow type specified, return None, otherwise, return a list of required but missing properties. Nz5No available properties had been set, skipping check.zBLoaded bundle does not contain the following required properties: FrazGLoaded bundle defines the following optional properties with wrong ID: ) rdr{r@r4rBrzrtrrq_check_optional_idappendextend)rEretZ wrong_propsrwrxrfrIrJr{s     zConfigWorkflow.check_propertiesrrMr[ list[Any]cKsg}||jvrMt|j|tr9tt|j|D]}|t|}||jj|fi||jj |q|S||jj|fi||jj ||S)z Evaluate the expression or expression list given by `id`. The resolved values from the evaluations are not stored, allowing this to be evaluated repeatedly (eg. in streaming applications) without restarting the hosting process. ) rr6r9rangelenrrget_parsed_content ref_resolverremove_resolved_content)rErrMrisub_idrIrIrJr s zConfigWorkflow._run_exprrYrZcCs@|tj}||jvr|tjdsdStd|d|d|S)NFz Property 'z' with config ID 'z' not in the config.)rrrrtrrqrCrErYrZprop_idrIrIrJ _get_prop_ids  zConfigWorkflow._get_prop_idcCs4|jstd|||}|dur|jj|dSdS)a With specified property name and information, get the parsed property value from config. Args: name: the name of target property. property: other information for the target property, defined in `TrainProperties` or `InferProperties`. z>Please execute 'initialize' before getting any parsed content.Nr)rrrrrrrIrIrJr\$s  zConfigWorkflow._get_propertyr]cCs8|||}|dur||j|<d|_|jjdSdS)r^NF)rrrrr)rErYrZr]rrIrIrJr`2s  zConfigWorkflow._set_propertyrm config_idrncs&tj|||d||j|tj<dS)ao Besides the default predefined properties, some 3rd party applications may need the bundle definition to provide additional properties for the specific use cases, if the bundle can't provide the property, means it can't work with the application. This utility adds the property for the application requirements check and access. Args: name: the name of target property. required: whether the property is "must-have". config_id: the config ID of target property in the bundle definition. desc: descriptions for the property. )rYrmrnN)rdrrr@rr)rErYrmrrnrfrIrJrrCszConfigWorkflow.add_propertyboolcCs|tjd}|tjd}|durdSd}|dvrr0rrabcrrcollections.abcrrZlogging.configrpathlibrtypingr monai.apps.utilsr monai.bundle.config_parserr Zmonai.bundle.propertiesr r rmonai.bundle.utilsrrrr monai.configr monai.utilsrrr__all__rr4rrrrIrIrIrJs.          Ov