U PhoY@sddlmZddlZddlZddlmZddlmZddlm Z ddl m Z m Z ddl mZmZmZmZddlmZdd lmZmZmZdd lmZdd lmZmZmZdd lmZm Z e rddl!Z!n ed \Z!Z"dgZ#dddddZ$GdddZ%dS)) annotationsN)Sequence)deepcopy)Path) TYPE_CHECKINGAny)ComponentLocatorConfigComponentConfigExpression ConfigItem)ReferenceResolver) ID_REF_KEY ID_SEP_KEY MACRO_KEY)PathLike) ensure_tuplelook_up_optionoptional_import)CheckKeyDuplicatesYamlLoadercheck_key_duplicatesyaml ConfigParsermonaitorchnumpy)rrnprc @seZdZdZdZddedZdedZe de de de d Z d ZdUd d ddddZddZddZdd dddZdd ddddZdVdd d d!d"d#ZdWd dd%dd&d'd(Zd)dd*d+d,Zdd%dd-d.ZdXd%dd/d0d1ZdYdd d d2d3d4Zd5d dd6d7d8Zd5d dd6d9d:ZdZd dd d;dd?Zd[d ddd;d@dAZedBd dCdDdEdFZ ed5d dCdGdHdIZ!ed\dCdBdd ddKdLdMZ"eddNdOdPdQZ#eddddRdSdTZ$d S)]ra The primary configuration parser. It traverses a structured config (in the form of nested Python dict or list), creates ``ConfigItem``, and assign unique IDs according to the structures. This class provides convenient access to the set of ``ConfigItem`` of the config by ID. A typical workflow of config parsing is as follows: - Initialize ``ConfigParser`` with the ``config`` source. - Call ``get_parsed_content()`` to get expected component with `id`. .. code-block:: python from monai.bundle import ConfigParser config = { "my_dims": 2, "dims_1": "$@my_dims + 1", "my_xform": {"_target_": "LoadImage"}, "my_net": {"_target_": "BasicUNet", "spatial_dims": "@dims_1", "in_channels": 1, "out_channels": 4}, "trainer": {"_target_": "SupervisedTrainer", "network": "@my_net", "preprocessing": "@my_xform"} } # in the example $@my_dims + 1 is an expression, which adds 1 to the value of @my_dims parser = ConfigParser(config) # get/set configuration content, the set method should happen before calling parse() print(parser["my_net"]["in_channels"]) # original input channels 1 parser["my_net"]["in_channels"] = 4 # change input channels to 4 print(parser["my_net"]["in_channels"]) # instantiate the network component parser.parse(True) net = parser.get_parsed_content("my_net", instantiate=True) print(net) # also support to get the configuration content of parsed `ConfigItem` trainer = parser.get_parsed_content("trainer", instantiate=False) print(trainer) Args: config: input config source to parse. excludes: when importing modules to instantiate components, excluding components from modules specified in ``excludes``. globals: pre-import packages as global variables to ``ConfigExpression``, so that expressions, for example, ``"$monai.data.list_data_collate"`` can use ``monai`` modules. The current supported globals and alias names are ``{"monai": "monai", "torch": "torch", "np": "numpy", "numpy": "numpy"}``. These are MONAI's minimal dependencies. Additional packages could be included with `globals={"itk": "itk"}`. Set it to ``False`` to disable `self.globals` module importing. See also: - :py:class:`monai.bundle.ConfigItem` - :py:class:`monai.bundle.scripts.run` )jsonrymlz.*\.(|)(z$)z(?:+Z_meta_NrzSequence[str] | str | Nonezdict[str, Any] | None | bool)configexcludesglobalscCsd|_i|_t}t|tr0|dkr0|||dk rr|dk rr|D](\}}t|trft |dn||j|<qHt |d|_ t |_ |dkr|jii}|j|ddS)N)NFFr)r#r")r"r$_default_globalscopy isinstancedictupdateitemsstrrrlocatorr ref_resolvermeta_keyset)selfr"r#r$_globalskvr5O/home/dell461/cl/sdc2/HISourceFinder-master-l/src/monai/bundle/config_parser.py__init__fs "  zConfigParser.__init__cCs|jS)Nr%r1r5r5r6__repr__{szConfigParser.__repr__cCs ||S)a, Get the parsed result of ``ConfigItem`` with the specified ``id`` with default arguments (e.g. ``lazy=True``, ``instantiate=True`` and ``eval_expr=True``). Args: id: id of the ``ConfigItem``. See also: :py:meth:`get_parsed_content` )get_parsed_content)r1idr5r5r6 __getattr__~s zConfigParser.__getattr__z str | int)r;returnc Cs|dkr|jS|j}t|D]}t|ttfsPtd|dt|d|dz(t|trjt||ddn |t |}Wqtk r}zt d||W5d }~XYqXq|S) a Get the config by id. Args: id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. z%config must be dict or list for key `z `, but got z: .F)print_all_optionsz query key: N) r"r split_idr(r)list ValueErrortyperintKeyError)r1r;r"r3er5r5r6 __getitem__s  "$zConfigParser.__getitem__None)r;r"r=cCsb|dkr||_|jdStj|dd\}}||}t|trD|nt|}|||<|jdS)at Set config by ``id``. Note that this method should be used before ``parse()`` or ``get_parsed_content()`` to ensure the updates are included in the parsed content. Args: id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. config: config to set at location ``id``. r>NT)last)r"r.resetr rAr(r)rE)r1r;r"Zlast_idbase_idconf_indexingr5r5r6 __setitem__s   zConfigParser.__setitem__r>r,z Any | None)r;defaultr=c Cs.z ||WStttfk r(|YSXdS)z Get the config by id. Args: id: id to specify the expected position. See also :py:meth:`__getitem__`. default: default value to return if the specified ``id`` is invalid. NrF IndexErrorrC)r1r;rPr5r5r6gets  zConfigParser.getTbool)r"r; recursiver=cCst|}|}|rn|dkr(i|_}|ddD]8}t|trR||krRi||<|t|trb|nt|}q4||t|<dS)a Set config by ``id``. Args: config: config to set at location ``id``. id: id to specify the expected position. See also :py:meth:`__setitem__`. recursive: if the nested id doesn't exist, whether to recursively create the nested items in the config. default to `True`. for the nested id, only support `dict` for the missing section. N)r rArSr"r(r)rE normalize_id)r1r"r;rUkeysrMr3r5r5r6r0s  zConfigParser.setzdict[str, Any])pairsr=cCs|D]\}}|||<qdS)a  Set the ``id`` and the corresponding config content in pairs, see also :py:meth:`__setitem__`. For example, ``parser.update({"train::epoch": 100, "train::lr": 0.02})`` Args: pairs: dictionary of `id` and config pairs. N)r+)r1rYr3r4r5r5r6r*s zConfigParser.updatec Cs0z||}WdStttfk r*YdSXdS)z Returns True if `id` is stored in this configuration. Args: id: id to specify the expected position. See also :py:meth:`__getitem__`. TFNrQ)r1r;_r5r5r6 __contains__s zConfigParser.__contains__)rKr=cCs*|r|j||j|ddS)aE Recursively resolve `self.config` to replace the macro tokens with target content. Then recursively parse the config source, add every item as ``ConfigItem`` to the reference resolver. Args: reset: whether to reset the ``reference_resolver`` before parsing. Defaults to `True`. r%N)r.rKresolve_macro_and_relative_ids _do_parserS)r1rKr5r5r6parses  zConfigParser.parse)r;kwargsr=cKsP|js|jddn"|dds:|j|dd d|jjfd|i|S)a Get the parsed result of ``ConfigItem`` with the specified ``id``. - If the item is ``ConfigComponent`` and ``instantiate=True``, the result is the instance. - If the item is ``ConfigExpression`` and ``eval_expr=True``, the result is the evaluated output. - Else, the result is the configuration content of `ConfigItem`. Args: id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. kwargs: additional keyword arguments to be passed to ``_resolve_one_item``. Currently support ``lazy`` (whether to retain the current config cache, default to `True`), ``instantiate`` (whether to instantiate the `ConfigComponent`, default to `True`) and ``eval_expr`` (whether to evaluate the `ConfigExpression`, default to `True`), ``default`` (the default config item if the `id` is not in the config content). T)rKlazyr;)r.Z is_resolvedr^rSZget_resolved_content)r1r;r_r5r5r6r: s   zConfigParser.get_parsed_contentz$PathLike | Sequence[PathLike] | dict)fr_r=cKs||j|f||jdS)a Read the metadata from specified JSON or YAML file. The metadata as a dictionary will be stored at ``self.config["_meta_"]``. Args: f: filepath of the metadata file, the content must be a dictionary, if providing a list of files, will merge the content of them. if providing a dictionary directly, use it as metadata. kwargs: other arguments for ``json.load`` or ``yaml.safe_load``, depends on the file format. N)r0load_config_filesr/)r1rar_r5r5r6 read_meta$s zConfigParser.read_metacKs8|j||jii}||j|f||j|ddS)a Read the config from specified JSON/YAML file or a dictionary and override the config content in the `self.config` dictionary. Args: f: filepath of the config file, the content must be a dictionary, if providing a list of files, wil merge the content of them. if providing a dictionary directly, use it as config. kwargs: other arguments for ``json.load`` or ``yaml.safe_load``, depends on the file format. r%N)r/rSr*rbr0)r1rar_contentr5r5r6 read_config2s zConfigParser.read_config)r"r;r=c Cst|ttfr:|jj||dD]\}}}|||||<qt|tr|||}|t rt |t t d\}}t |s| nt |d}t||S|S)a^ Recursively resolve `self.config` to replace the relative ids with absolute ids, for example, `@##A` means `A` in the upper level. and replace the macro tokens with target content, The macro tokens start with "%", can be from another structured file, like: ``"%default_net"``, ``"%/data/config.json#net"``. Note that the macro replacement doesn't support recursive macro tokens. Args: config: input config file to resolve. id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. r;r"Nr%)r(r)rBr.iter_subconfigs _do_resolver,resolve_relative_ids startswithrr split_path_idlenrSload_config_filer) r1r"r;r3sub_idr4pathidsparserr5r5r6rhBs    zConfigParser._do_resolvecCs||j|ddS)ao Recursively resolve `self.config` to replace the relative ids with absolute ids, for example, `@##A` means `A` in the upper level. and replace the macro tokens with target content, The macro tokens are marked as starting with "%", can be from another structured file, like: ``"%default_net"``, ``"%/data/config.json::net"``. r%N)r0rhrSr8r5r5r6r\^sz+ConfigParser.resolve_macro_and_relative_idscCst|ttfr8|jj||dD]\}}}|j||dqt|r\|jt|||j dn8t |r|jt |||j dn|jt ||ddS)a! Recursively parse the nested data in config source, add every item as `ConfigItem` to the resolver. Args: config: config source to parse. id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. rf)r"r;)r"r;r-)r"r;r$N)r(r)rBr.rgr]r is_instantiableZadd_itemr-r is_expressionr$r )r1r"r;rZrnr4r5r5r6r]hs   zConfigParser._do_parserr))filepathr_r=c Ks|siStt|}t|jtj|s:td|dt|}| |j drzt j |fdti|W5QRS| |j ddrtj |tf|W5QRStd|dW5QRXdS) aI Load a single config file with specified file path (currently support JSON and YAML files). Args: filepath: path of target file to load, supported postfixes: `.json`, `.yml`, `.yaml`. kwargs: other arguments for ``json.load`` or ```yaml.safe_load``, depends on the file format. zunknown file input: ""robject_pairs_hookNz7only support JSON or YAML config file so far, got name r?)r,rrecompile path_match IGNORECASEfindallrCopenlowerendswithsuffixesrloadrrr)clsrtr_ _filepathrar5r5r6rms   "zConfigParser.load_config_file)filesr_r=cKszt|tr|Stid}t|tr@t|s@d|kr@|d}t|D](}|j|f| D]\}}|||<q^qH| S)a Load multiple config files into a single config dict. The latter config file in the list will override or add the former config file. ``"::"`` (or ``"#"``) in the config keys are interpreted as special characters to go one level further into the nested structures. Args: files: path of target files to load, supported postfixes: `.json`, `.yml`, `.yaml`. if providing a list of files, will merge the content of them. if providing a string with comma separated file paths, will merge the content of them. if providing a dictionary, return it directly. kwargs: other arguments for ``json.load`` or ```yaml.safe_load``, depends on the file format. r%,) r(r)rr,ris_filesplitrrmr+rS)rrr_rqir3r4r5r5r6rbs     zConfigParser.load_config_filesr)r"rtfmtr_r=c Kstt|}t|dddh}t|dh}|dkrRtj||f|W5QRdS|dksb|dkr~tj||f|W5QRSt d|dW5QRXdS)a Export the config content to the specified file path (currently support JSON and YAML files). Args: config: source config content to export. filepath: target file path to save. fmt: format of config content, currently support ``"json"`` and ``"yaml"``. kwargs: other arguments for ``json.dump`` or ``yaml.safe_dump``, depends on the file format. rrrwNz2only support JSON or YAML config file so far, got r?) r,rrr~r}rdumpr safe_dumprC)rr"rtrr_rwriterrar5r5r6export_config_files  zConfigParser.export_config_fileztuple[str, str])srcr=cCsxt|}td|jdtdtj|}|s:d|fS|dd}||d\}}|| trr|t tdndfS)a. Split `src` string into two parts: a config file path and component id. The file path should end with `(json|yaml|yml)`. The component id should be separated by `::` if it exists. If no path or no id, return "". Args: src: source string to split. r z(?=(?:z.*)|$))r>rrwN) r rWrxry suffix_matchrr{r|rsplitrjrl)rrresult path_namerZrpr5r5r6rks $ zConfigParser.split_path_id)r;valuer=c Cst|}tt|j|dd}|t}|D]}t |krDt nt }|t |d t}|t |krzt d|d|t |krd}nt|d| t}||||}q4|S)a" To simplify the reference or macro tokens ID in the nested config content, it's available to use relative ID name which starts with the `ID_SEP_KEY`, for example, "@#A" means `A` in the same level, `@##A` means `A` in the upper level. It resolves the relative ids to absolute ids. For example, if the input data is: .. code-block:: python { "A": 1, "B": {"key": "@##A", "value1": 2, "value2": "%#value1", "value3": [3, 4, "@#1"]}, } It will resolve `B` to `{"key": "@A", "value1": 2, "value2": "%B#value1", "value3": [3, 4, "@B#value3#1"]}`. Args: id: id name for current config item to compute relative id. value: input value to resolve relative ids. T)reverseNzthe relative id in `z(` is out of the range of config content.r>)r rWsortedr0unionrelative_id_prefixr|rrr rrlcountrCjoinreplace) rr;rprefixesZ current_idpsymlengthnewr5r5r6ris    z!ConfigParser.resolve_relative_ids)NNN)r>N)r>T)T)r>)r>)r>)r)%__name__ __module__ __qualname____doc__rrrrzrxryr rrrr/r7r9r<rHrOrSr0r*r[r^r:rcrerhr\r] classmethodrmrbrrkrir5r5r5r6r&sF8     )& __future__rrrxcollections.abcrr'rpathlibrtypingrrZmonai.bundle.config_itemrr r r Zmonai.bundle.reference_resolverr monai.bundle.utilsr rr monai.configr monai.utilsrrrmonai.utils.miscrrrrZ__all__r&rr5r5r5r6 s$