o  i3A@sddlmZddlZddlmZddlmZddlmZm Z ddl m Z ddl m Z ddlmZdd lmZmZmZmZdd lmZdd lmZdd lmZdd lmZed\ZZed\ZZ ee!dZ"gdZ#GdddeZ$Gddde$Z%Gddde$Z&dS)) annotationsN)abstractmethod)deepcopy)Anycast)warn) BundleAlgo) get_logger)AlgoAlgoGenalgo_from_picklealgo_to_pickle) ConfigParser)PathLike)optional_import)AlgoKeysnnioptuna) module_name)HPOGenNNIGen OptunaGenc@s@eZdZdZeddZeddZeddZedd Zd S) ra> The base class for hyperparameter optimization (HPO) interfaces to generate algos in the Auto3Dseg pipeline. The auto-generated algos are saved at their ``output_path`` on the disk. The files in the ``output_path`` may contain scripts that define the algo, configuration files, and pickle files that save the internal states of the algo before/after the training. Compared to the BundleGen class, HPOGen generates Algo on-the-fly, so training and algo generation may be executed alternatively and take a long time to finish the generation process. cCt)z Get the hyperparameter from HPO.NotImplementedErrorselfr^/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/apps/auto3dseg/hpo_gen.pyget_hyperparameters-zHPOGen.get_hyperparameterscOr)zHUpdate Algo parameters according to the hyperparameters to be evaluated.rrargskwargsrrr update_params2r zHPOGen.update_paramscOr)z$Report the evaluated results to HPO.rr!rrr set_score7r zHPOGen.set_scorecOr)zDInterface for launch the training given the fetched hyperparameters.rr!rrrrun_algo<r zHPOGen.run_algoN) __name__ __module__ __qualname____doc__rrr$r%r&rrrrr#s   rc@sfeZdZdZd"d#ddZd d Zd d Zd dZd$ddZddZ d%d&ddZ ddZ d'd(d d!Z dS))ru Generate algorithms for the NNI to automate hyperparameter tuning. The module has two major interfaces: ``__init__`` which prints out how to set up the NNI, and a trialCommand function ``run_algo`` for the NNI library to start the trial of the algo. More about trialCommand function can be found in ``trail code`` section in NNI webpage https://nni.readthedocs.io/en/latest/tutorials/hpo_quickstart_pytorch/main.html . Args: algo: an Algo object (e.g. BundleAlgo) with defined methods: ``get_output_path`` and train and supports saving to and loading from pickle files via ``algo_from_pickle`` and ``algo_to_pickle``. params: a set of parameter to override the algo if override is supported by Algo subclass. Examples:: The experiment will keep generating new folders to save the model checkpoints, scripts, and configs if available. ├── algorithm_templates │ └── unet ├── unet_0 │ ├── algo_object.pkl │ ├── configs │ └── scripts ├── unet_0_learning_rate_0.01 │ ├── algo_object.pkl │ ├── configs │ ├── model_fold0 │ └── scripts └── unet_0_learning_rate_0.1 ├── algo_object.pkl ├── configs ├── model_fold0 └── scripts .. code-block:: python # Bundle Algorithms are already generated by BundleGen in work_dir import_bundle_algo_history(work_dir, only_trained=False) algo_dict = self.history[0] # pick the first algorithm algo_name = algo_dict[AlgoKeys.ID] onealgo = algo_dict[AlgoKeys.ALGO] nni_gen = NNIGen(algo=onealgo) nni_gen.print_bundle_algo_instruction() Notes: The NNIGen will prepare the algorithms in a folder and suggest a command to replace trialCommand in the experiment config. However, NNIGen will not trigger NNI. User needs to write their NNI experiment configs, and then run the NNI command manually. Nalgo Algo | Noneparams dict | NonecCs|d|_d|_|durRt|trC|dur||_n-t||_tj| d}tj | }| ddi|jj ||fi|n||_t |j|jjd|_dSdSN _overridefill_with_datastatsF template_path)hint obj_filename isinstancerr+rospathbasenameget_output_pathdirnameupdateexport_to_diskr r4rr+r-name output_folderrrr__init__qs  zNNIGen.__init__cC|jS)z5Return the filename of the dumped pickle algo object.r6rrrrget_obj_filenamezNNIGen.get_obj_filenamecCsd}tdtdtdt|d|jdtdtdtd tt|jjd td |jd td t|dtddS)zH Print how to write the trial commands for Bundle Algo. z/python -m monai.apps.auto3dseg NNIGen run_algo z============================================================================================================================================z#If NNI will run in your local env: zB1. Add the following line to the trialCommand in your NNI config:  z {result_dir}z--------------------------------------------------------------------------------------------------------------------------------------------z!If NNI will run in a remote env: z'1. Copy the algorithm_templates folder z+ to remote {remote_algorithm_templates_dir}z2. Copy the older z( to the remote machine {remote_algo_dir}zDThen add the following line to the trialCommand in your NNI config: z@ {remote_algo_dir} {result_dir} {remote_algorithm_templates_dir}N)loggerinfor6rrr+r4r;)rr5rrrprint_bundle_algo_instructions      z$NNIGen.print_bundle_algo_instructioncCstrtStdiS)zK Get parameter for next round of training from NNI server. ?NNI is not detected. The code will continue to run without NNI.)has_nnirZget_next_parameterrrrrrrszNNIGen.get_hyperparametersdictreturnNonecC ||_dS)z Translate the parameter from monai bundle to meet NNI requirements. Args: params: a dict of parameters. Nr-rr-rrrr$ zNNIGen.update_paramscCddd|jDpdS) Get the identifier of the current experiment. In the format of listing the searching parameter name and values connected by underscore in the file name. r0cs$|] \}}d|d|VqdS_Nr.0kvrrr "z%NNIGen.get_task_id.._Nonejoinr-itemsrrrr get_task_idzNNIGen.get_task_id.rAstrcCs|}tj|j}tj|||}tj|d|_t|jt r2|jj ||||dddSt |j |t|dS) Generate the record for each Algo. If it is a BundleAlgo, it will generate the config files. Args: output_folder: the directory nni will save the results to. algo_object.pklF) bundle_rootr2Nrcr8r9r:r+r;rar6r7rr>rexport_config_filer-rHrIrrAZtask_idZ task_prefix write_pathrrrgenerates   zNNIGen.generatecCstr t|dStddS)z/ Report the acc to NNI server. rKN)rLrZreport_final_resultrraccrrrr%s zNNIGen.set_scorer6r4PathLike | NonecCtj|s t|dt||d\|_}|}|||||j |j |j }t t j|i}t|jfd|jji|||dSam The python interface for NNI to run. Args: obj_filename: the pickle-exported Algo object. output_folder: the root path of the algorithms templates. template_path: the algorithm_template. It must contain algo.py in the follow path: ``{algorithm_templates_dir}/{network}/scripts/algo.py`` z is not foundr3r4Nr8r9isfile ValueErrorr r+rr$rntrainr- get_scorerfrSCOREr r4r%rr6rAr4algo_meta_datar-rprrrr&s    zNNIGen.run_algoNN)r+r,r-r.r-rMrNrOrerArfrNrOreNr6rfrArfr4rqrNrO) r'r(r)r*rBrErJrr$rcrnr%r&rrrrrBs .   rc@steZdZdZd&d'd d Zd d Zd dZddZddZ d(d)ddZ d*ddZ d d!Z d+d,d"d#Z d(d-d$d%Z dS).ru Generate algorithms for the Optuna to automate hyperparameter tuning. Please refer to NNI and Optuna (https://optuna.readthedocs.io/en/stable/) for more information. Optuna has different running scheme compared to NNI. The hyperparameter samples come from a trial object (trial.suggest...) created by Optuna, so OptunaGen needs to accept this trial object as input. Meanwhile, Optuna calls OptunaGen, thus OptunaGen.__call__() should return the accuracy. Use functools.partial to wrap OptunaGen for addition input arguments. Args: algo: an Algo object (e.g. BundleAlgo). The object must at least define two methods: get_output_path and train and supports saving to and loading from pickle files via ``algo_from_pickle`` and ``algo_to_pickle``. params: a set of parameter to override the algo if override is supported by Algo subclass. Examples:: The experiment will keep generating new folders to save the model checkpoints, scripts, and configs if available. ├── algorithm_templates │ └── unet ├── unet_0 │ ├── algo_object.pkl │ ├── configs │ └── scripts ├── unet_0_learning_rate_0.01 │ ├── algo_object.pkl │ ├── configs │ ├── model_fold0 │ └── scripts └── unet_0_learning_rate_0.1 ├── algo_object.pkl ├── configs ├── model_fold0 └── scripts Notes: Different from NNI and NNIGen, OptunaGen and Optuna can be ran within the Python process. Nr+r,r-r.rNrOcCs|d|_|durOt|tr@|dur||_n-t||_tj|d}tj |}| ddi|jj ||fi|n||_t |j|jj d|_dSdSr/)r6r7rr+rr8r9r:r;r<r=r>r r4r?rrrrBs  zOptunaGen.__init__cCrC)z(Return the dumped pickle object of algo.rDrrrrrE-rFzOptunaGen.get_obj_filenamecCs.trtdd|jdddiStdiS)z Get parameter for next round of training from optuna trial object. This function requires user rewrite during usage for different search space. z2Please rewrite this code by creating a child classZ learning_rateg-C6?g?zEOptuna is not detected. The code will continue to run without Optuna.) has_optunarHrItrialZ suggest_floatrrrrrr1s  zOptunaGen.get_hyperparameterscCrP)zSet the accuracy scoreN)rprorrrr%= zOptunaGen.set_scorecCrP)zSet the Optuna trialN)r)rrrrr set_trialArzOptunaGen.set_trialrerrr6rfrAr4rqcCs|||||||jS)a Callable that Optuna will use to optimize the hyper-parameters Args: obj_filename: the pickle-exported Algo object. output_folder: the root path of the algorithms templates. template_path: the algorithm_template. It must contain algo.py in the follow path: ``{algorithm_templates_dir}/{network}/scripts/algo.py`` )rr&rp)rrr6rAr4rrr__call__Es zOptunaGen.__call__rMcCrP)zu Translate the parameter from monai bundle. Args: params: a dict of parameters. NrQrRrrrr$UrSzOptunaGen.update_paramscCrT)rUr0csrVrWrrYrrrr]cr^z(OptunaGen.get_task_id..r_r`rrrrrc^rdzOptunaGen.get_task_idcCs~|}tj|j}tj|||}tj|d|_t|jt r1|jj |||dddSt |j |t|dS)rgrhF)r2Nrjrlrrrrnes zOptunaGen.generatecCrrrsrtrzrrrr&ws    zOptunaGen.run_algor|)r+r,r-r.rNrOr) rrr6rfrArfr4rqrNrr}r~rr)r'r(r)r*rBrErr%rrr$rcrnr&rrrrrs &   r)' __future__rr8abcrcopyrtypingrrwarningsrmonai.apps.auto3dseg.bundle_genrmonai.apps.utilsr monai.auto3dsegr r r r monai.bundle.config_parserr monai.configr monai.utilsrmonai.utils.enumsrrrLrrr'rH__all__rrrrrrrs*             1