o $ iY@sddlmZddlZddlZddlmZddlmZddlm Z ddl m Z m Z ddl mZmZmZmZddlmZdd lmZmZmZmZdd lmZdd lmZmZmZdd lm Z m!Z!e reddl"Z"ned \Z"Z#dgZ$dddddZ%GdddZ&dS)) annotationsN)Sequence)deepcopy)Path) TYPE_CHECKINGAny)ComponentLocatorConfigComponentConfigExpression ConfigItem)ReferenceResolver) ID_REF_KEY ID_SEP_KEY MACRO_KEYmerge_kv)PathLike) ensure_tuplelook_up_optionoptional_import)CheckKeyDuplicatesYamlLoadercheck_key_duplicatesyaml ConfigParsermonaitorchnumpy)rrnprc @sJeZdZdZdZddedZdedZe de de de d Z d Z dVdWddZddZddZdXddZdYddZdZd[d$d%Zd\d]d)d*Zd^d-d.Zd_d/d0Zd`dad2d3Zdbdcd5d6Zddd9d:Zddd;d<Zdbded=d>Zd?d@ZdbdfdAdBZedgdFdGZ edhdIdJZ!edidjdMdNZ"edkdQdRZ#edldTdUZ$d S)mra The primary configuration parser. It traverses a structured config (in the form of nested Python dict or list), creates ``ConfigItem``, and assign unique IDs according to the structures. This class provides convenient access to the set of ``ConfigItem`` of the config by ID. A typical workflow of config parsing is as follows: - Initialize ``ConfigParser`` with the ``config`` source. - Call ``get_parsed_content()`` to get expected component with `id`. .. code-block:: python from monai.bundle import ConfigParser config = { "my_dims": 2, "dims_1": "$@my_dims + 1", "my_xform": {"_target_": "LoadImage"}, "my_net": {"_target_": "BasicUNet", "spatial_dims": "@dims_1", "in_channels": 1, "out_channels": 4}, "trainer": {"_target_": "SupervisedTrainer", "network": "@my_net", "preprocessing": "@my_xform"} } # in the example $@my_dims + 1 is an expression, which adds 1 to the value of @my_dims parser = ConfigParser(config) # get/set configuration content, the set method should happen before calling parse() print(parser["my_net"]["in_channels"]) # original input channels 1 parser["my_net"]["in_channels"] = 4 # change input channels to 4 print(parser["my_net"]["in_channels"]) # instantiate the network component parser.parse(True) net = parser.get_parsed_content("my_net", instantiate=True) print(net) # also support to get the configuration content of parsed `ConfigItem` trainer = parser.get_parsed_content("trainer", instantiate=False) print(trainer) Args: config: input config source to parse. excludes: when importing modules to instantiate components, excluding components from modules specified in ``excludes``. globals: pre-import packages as global variables to ``ConfigExpression``, so that expressions, for example, ``"$monai.data.list_data_collate"`` can use ``monai`` modules. The current supported globals and alias names are ``{"monai": "monai", "torch": "torch", "np": "numpy", "numpy": "numpy"}``. These are MONAI's minimal dependencies. Additional packages could be included with `globals={"itk": "itk"}`. Set it to ``False`` to disable `self.globals` module importing. See also: - :py:class:`monai.bundle.ConfigItem` - :py:class:`monai.bundle.scripts.run` )jsonrymlz.*\.(|)(z$)z(?:+Z_meta_NconfigrexcludesSequence[str] | str | Noneglobalsdict[str, Any] | None | boolcCsd|_i|_t}t|tr|dvr|||dur9|dur9|D]\}}t|tr3t |dn||j|<q$t |d|_ t |_ |durL|jii}|j|j |ddS)N)NFFr)r$r#)r#r&_default_globalscopy isinstancedictupdateitemsstrrrlocatorr ref_resolvermeta_keysetnormalize_meta_id)selfr#r$r&_globalskvr9\/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/bundle/config_parser.py__init__fs "  zConfigParser.__init__cCs|jS)Nr(r5r9r9r:__repr__{szConfigParser.__repr__cCs ||S)a, Get the parsed result of ``ConfigItem`` with the specified ``id`` with default arguments (e.g. ``lazy=True``, ``instantiate=True`` and ``eval_expr=True``). Args: id: id of the ``ConfigItem``. See also: :py:meth:`get_parsed_content` )get_parsed_content)r5idr9r9r: __getattr__~s zConfigParser.__getattr__r? str | intreturnc Cs|dkr|jS|j}t|D]A}t|ttfs(td|dt|d|dzt|tr5t||ddn|t |}WqtyP}zt d||d }~ww|S) a Get the config by id. Args: id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. z%config must be dict or list for key `z `, but got z: .F)print_all_optionsz query key: N) r#r split_idr+r,list ValueErrortyperintKeyError)r5r?r#r7er9r9r: __getitem__s  "zConfigParser.__getitem__NonecCsb|dkr||_|jdStj|dd\}}||}t|tr"|nt|}|||<|jdS)at Set config by ``id``. Note that this method should be used before ``parse()`` or ``get_parsed_content()`` to ensure the updates are included in the parsed content. Args: id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. config: config to set at location ``id``. rCNT)last)r#r1resetr rFr+r,rJ)r5r?r#Zlast_idbase_idconf_indexingr9r9r: __setitem__s   zConfigParser.__setitem__rCr/default Any | Nonec Cs(z||WStttfy|YSw)z Get the config by id. Args: id: id to specify the expected position. See also :py:meth:`__getitem__`. default: default value to return if the specified ``id`` is invalid. rK IndexErrorrH)r5r?rUr9r9r:gets  zConfigParser.getT recursiveboolcCst|}|}|r7|duri|_}|ddD]}t|tr)||vr)i||<|t|tr1|nt|}q|j||t |<dS)a Set config by ``id``. Args: config: config to set at location ``id``. id: id to specify the expected position. See also :py:meth:`__setitem__`. recursive: if the nested id doesn't exist, whether to recursively create the nested items in the config. default to `True`. for the nested id, only support `dict` for the missing section. N) r rFrYr#r+r,rJr1r4 normalize_id)r5r#r?rZkeysrRr7r9r9r:r3s  zConfigParser.setpairsdict[str, Any]cCs|D]\}}|||<qdS)a  Set the ``id`` and the corresponding config content in pairs, see also :py:meth:`__setitem__`. For example, ``parser.update({"train::epoch": 100, "train::lr": 0.02})`` Args: pairs: dictionary of `id` and config pairs. N)r.)r5r_r7r8r9r9r:r-s zConfigParser.updatec Cs*z||}WdStttfyYdSw)z Returns True if `id` is stored in this configuration. Args: id: id to specify the expected position. See also :py:meth:`__getitem__`. TFrW)r5r?_r9r9r: __contains__s zConfigParser.__contains__rPcCs*|r|j||j|ddS)aE Recursively resolve `self.config` to replace the macro tokens with target content. Then recursively parse the config source, add every item as ``ConfigItem`` to the reference resolver. Args: reset: whether to reset the ``reference_resolver`` before parsing. Defaults to `True`. r(N)r1rPresolve_macro_and_relative_ids _do_parserY)r5rPr9r9r:parses zConfigParser.parsekwargscKsP|js |jddn|dds|j|dd d|jjdd|i|S)a Get the parsed result of ``ConfigItem`` with the specified ``id``. - If the item is ``ConfigComponent`` and ``instantiate=True``, the result is the instance. - If the item is ``ConfigExpression`` and ``eval_expr=True``, the result is the evaluated output. - Else, the result is the configuration content of `ConfigItem`. Args: id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. kwargs: additional keyword arguments to be passed to ``_resolve_one_item``. Currently support ``lazy`` (whether to retain the current config cache, default to `True`), ``instantiate`` (whether to instantiate the `ConfigComponent`, default to `True`) and ``eval_expr`` (whether to evaluate the `ConfigExpression`, default to `True`), ``default`` (the default config item if the `id` is not in the config content). T)rPlazyr?Nr9)r1Z is_resolvedrerYZget_resolved_content)r5r?rfr9r9r:r> s  zConfigParser.get_parsed_contentf$PathLike | Sequence[PathLike] | dictcKs ||j|fi||jdS)a Read the metadata from specified JSON or YAML file. The metadata as a dictionary will be stored at ``self.config["_meta_"]``. Args: f: filepath of the metadata file, the content must be a dictionary, if providing a list of files, will merge the content of them. if providing a dictionary directly, use it as metadata. kwargs: other arguments for ``json.load`` or ``yaml.safe_load``, depends on the file format. N)r3load_config_filesr2)r5rhrfr9r9r: read_meta$s zConfigParser.read_metacKs<|j||jii}||j|fi||j|ddS)a Read the config from specified JSON/YAML file or a dictionary and override the config content in the `self.config` dictionary. Args: f: filepath of the config file, the content must be a dictionary, if providing a list of files, wil merge the content of them. if providing a dictionary directly, use it as config. kwargs: other arguments for ``json.load`` or ``yaml.safe_load``, depends on the file format. r(N)r2rYr-rjr3)r5rhrfcontentr9r9r: read_config2s zConfigParser.read_configc Cst|ttfr|jj||dD] \}}}|||||<qt|trN|||}|t rNt |t t d\}}t |sA| nt |d}t||S|S)a^ Recursively resolve `self.config` to replace the relative ids with absolute ids, for example, `@##A` means `A` in the upper level. and replace the macro tokens with target content, The macro tokens start with "%", can be from another structured file, like: ``"%default_net"``, ``"%/data/config.json#net"``. Note that the macro replacement doesn't support recursive macro tokens. Args: config: input config file to resolve. id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. r?r#Nr()r+r,rGr1iter_subconfigs _do_resolver/resolve_relative_ids startswithrr split_path_idlenrYload_config_filer) r5r#r?r7sub_idr8pathidsparserr9r9r:rpBs    zConfigParser._do_resolvecCs||j|ddS)ao Recursively resolve `self.config` to replace the relative ids with absolute ids, for example, `@##A` means `A` in the upper level. and replace the macro tokens with target content, The macro tokens are marked as starting with "%", can be from another structured file, like: ``"%default_net"``, ``"%/data/config.json::net"``. r(N)r3rprYr<r9r9r:rc^sz+ConfigParser.resolve_macro_and_relative_idscCst|ttfr|jj||dD] \}}}|j||dqt|r/|jt|||j ddSt |rB|jt |||j ddS|jt ||ddS)a! Recursively parse the nested data in config source, add every item as `ConfigItem` to the resolver. Args: config: config source to parse. id: id of the ``ConfigItem``, ``"::"`` (or ``"#"``) in id are interpreted as special characters to go one level further into the nested structures. Use digits indexing from "0" for list or other strings for dict. For example: ``"xform::5"``, ``"net::channels"``. ``""`` indicates the entire ``self.config``. rn)r#r?)r#r?r0)r#r?r&N)r+r,rGr1rordr is_instantiableZadd_itemr0r is_expressionr&r )r5r#r?rarvr8r9r9r:rdhs   zConfigParser._do_parsefilepathrr,cKs|siStt|}t|jtj|std|dt|B}| |j dr>t j |fdti|WdS| |j ddr[tj |tfi|WdStd|d1sfwYdS) aI Load a single config file with specified file path (currently support JSON and YAML files). Args: filepath: path of target file to load, supported postfixes: `.json`, `.yml`, `.yaml`. kwargs: other arguments for ``json.load`` or ```yaml.safe_load``, depends on the file format. zunknown file input: ""robject_pairs_hookNz7only support JSON or YAML config file so far, got name rD)r/rrecompile path_match IGNORECASEfindallrHopenlowerendswithsuffixesrloadrrr)clsr|rf _filepathrhr9r9r:rus   zConfigParser.load_config_filefilescKst|tr|Stid}t|tr t|s d|vr |d}t|D]}|j|fi|}| D] \}}t |||q3q$| S)a Load multiple config files into a single config dict. The latter config file in the list will override or add the former config file. ``"::"`` (or ``"#"``) in the config keys are interpreted as special characters to go one level further into the nested structures. Args: files: path of target files to load, supported postfixes: `.json`, `.yml`, `.yaml`. if providing a list of files, will merge the content of them. if providing a string with comma separated file paths, will merge the content of them. if providing a dictionary, return it directly. kwargs: other arguments for ``json.load`` or ```yaml.safe_load``, depends on the file format. r(,) r+r,rr/ris_filesplitrrur.rrY)rrrfryi config_dictr7r8r9r9r:rjs    zConfigParser.load_config_filesrfmtcKstt|}t|hd}t|d9}|dkr,tj||fi| WddS|dks4|dkrEtj||fi|WdSt d|d1sPwYdS) a Export the config content to the specified file path (currently support JSON and YAML files). Args: config: source config content to export. filepath: target file path to save. fmt: format of config content, currently support ``"json"`` and ``"yaml"``. kwargs: other arguments for ``json.dump`` or ``yaml.safe_dump``, depends on the file format. >rrrwrNrrz2only support JSON or YAML config file so far, got rD) r/rrrrrdumpr safe_dumprH)rr#r|rrfrwriterrhr9r9r:export_config_files  zConfigParser.export_config_filesrctuple[str, str]cCszt|}td|jdtdtj|}|sd|fS|dd}||d\}}|| tr:|t tdfSdfS)a. Split `src` string into two parts: a config file path and component id. The file path should end with `(json|yaml|yml)`. The component id should be separated by `::` if it exists. If no path or no id, return "". Args: src: source string to split. r!z(?=(?:z.*)|$))rCrrN) r r]rr suffix_matchrrrrsplitrrrt)rrresult path_namerarxr9r9r:rss $ $zConfigParser.split_path_idvaluec Cst|}tt|j|dd}|t}|D]@}t |vr"t nt }|t |d t}|t |kr=t d|d|t |krFd}n t|d| t}||||}q|S)a" To simplify the reference or macro tokens ID in the nested config content, it's available to use relative ID name which starts with the `ID_SEP_KEY`, for example, "@#A" means `A` in the same level, `@##A` means `A` in the upper level. It resolves the relative ids to absolute ids. For example, if the input data is: .. code-block:: python { "A": 1, "B": {"key": "@##A", "value1": 2, "value2": "%#value1", "value3": [3, 4, "@#1"]}, } It will resolve `B` to `{"key": "@A", "value1": 2, "value2": "%B#value1", "value3": [3, 4, "@B#value3#1"]}`. Args: id: id name for current config item to compute relative id. value: input value to resolve relative ids. T)reverseNzthe relative id in `z(` is out of the range of config content.rC)r r]sortedr3unionrelative_id_prefixrrrr rrtcountrHjoinreplace) rr?rprefixesZ current_idpsymlengthnewr9r9r:rqs    z!ConfigParser.resolve_relative_ids)NNN)r#rr$r%r&r')r?rArBr)r?rAr#rrBrN)rCN)r?r/rUrVrBr)rCT)r#rr?r/rZr[rBrN)r_r`rBrN)r?rArBr[)T)rPr[rBrN)rC)r?r/rfrrBr)rhrirfrrBrN)r#rr?r/rBr)r#rr?r/rBrN)r|rrfrrBr,)rrirfrrBr,)r) r#r,r|rrr/rfrrBrN)rr/rBr)r?r/rr/rBr/)%__name__ __module__ __qualname____doc__rrrrrrr rrrr2r;r=r@rMrTrYr3r-rbrer>rkrmrprcrd classmethodrurjrrsrqr9r9r9r:r&sH8              )' __future__rrrcollections.abcrr*rpathlibrtypingrrZmonai.bundle.config_itemrr r r Zmonai.bundle.reference_resolverr monai.bundle.utilsr rrr monai.configr monai.utilsrrrmonai.utils.miscrrrra__all__r)rr9r9r9r:s&