U Ph"A@sddlmZddlZddlmZddlmZddlmZm Z ddl m Z ddl m Z ddlmZdd lmZmZmZmZdd lmZdd lmZdd lmZdd lmZed\ZZed\ZZ ee!dZ"dddgZ#GdddeZ$Gddde$Z%Gddde$Z&dS)) annotationsN)abstractmethod)deepcopy)Anycast)warn) BundleAlgo) get_logger)AlgoAlgoGenalgo_from_picklealgo_to_pickle) ConfigParser)PathLike)optional_import)AlgoKeysnnioptuna) module_nameHPOGenNNIGen OptunaGenc@s@eZdZdZeddZeddZeddZedd Zd S) ra> The base class for hyperparameter optimization (HPO) interfaces to generate algos in the Auto3Dseg pipeline. The auto-generated algos are saved at their ``output_path`` on the disk. The files in the ``output_path`` may contain scripts that define the algo, configuration files, and pickle files that save the internal states of the algo before/after the training. Compared to the BundleGen class, HPOGen generates Algo on-the-fly, so training and algo generation may be executed alternatively and take a long time to finish the generation process. cCstdS)z Get the hyperparameter from HPO.NNotImplementedErrorselfrQ/home/dell461/cl/sdc2/HISourceFinder-master-l/src/monai/apps/auto3dseg/hpo_gen.pyget_hyperparameters-szHPOGen.get_hyperparameterscOstdS)zHUpdate Algo parameters according to the hyperparameters to be evaluated.Nrrargskwargsrrr update_params2szHPOGen.update_paramscCstdS)z$Report the evaluated results to HPO.Nrrrrr set_score7szHPOGen.set_scorecOstdS)zDInterface for launch the training given the fetched hyperparameters.Nrrrrrrun_algo<szHPOGen.run_algoN) __name__ __module__ __qualname____doc__rrr"r#r$rrrrr#s    c@seZdZdZd dddddZdd Zd d Zd d ZdddddZddZ d!dddddZ ddZ d"dddddddZ dS)#ru Generate algorithms for the NNI to automate hyperparameter tuning. The module has two major interfaces: ``__init__`` which prints out how to set up the NNI, and a trialCommand function ``run_algo`` for the NNI library to start the trial of the algo. More about trialCommand function can be found in ``trail code`` section in NNI webpage https://nni.readthedocs.io/en/latest/tutorials/hpo_quickstart_pytorch/main.html . Args: algo: an Algo object (e.g. BundleAlgo) with defined methods: ``get_output_path`` and train and supports saving to and loading from pickle files via ``algo_from_pickle`` and ``algo_to_pickle``. params: a set of parameter to override the algo if override is supported by Algo subclass. Examples:: The experiment will keep generating new folders to save the model checkpoints, scripts, and configs if available. ├── algorithm_templates │ └── unet ├── unet_0 │ ├── algo_object.pkl │ ├── configs │ └── scripts ├── unet_0_learning_rate_0.01 │ ├── algo_object.pkl │ ├── configs │ ├── model_fold0 │ └── scripts └── unet_0_learning_rate_0.1 ├── algo_object.pkl ├── configs ├── model_fold0 └── scripts .. code-block:: python # Bundle Algorithms are already generated by BundleGen in work_dir import_bundle_algo_history(work_dir, only_trained=False) algo_dict = self.history[0] # pick the first algorithm algo_name = algo_dict[AlgoKeys.ID] onealgo = algo_dict[AlgoKeys.ALGO] nni_gen = NNIGen(algo=onealgo) nni_gen.print_bundle_algo_instruction() Notes: The NNIGen will prepare the algorithms in a folder and suggest a command to replace trialCommand in the experiment config. However, NNIGen will not trigger NNI. User needs to write their NNI experiment configs, and then run the NNI command manually. N Algo | None dict | None)algoparamscCs|d|_d|_|dk rt|tr|dkr2||_qt||_tj| d}tj | }| ddi|jj ||f|n||_t |j|jjd|_dSNZ _overridefill_with_datastatsF template_path)hint obj_filename isinstancerr+rospathbasenameget_output_pathdirnameupdateexport_to_diskr r1rr+r,name output_folderrrr__init__qs  zNNIGen.__init__cCs|jS)z5Return the filename of the dumped pickle algo object.r3rrrrget_obj_filenameszNNIGen.get_obj_filenamecCsd}tdtdtdt|d|jdtdtdtd tt|jjd td |jd td t|dtddS)zH Print how to write the trial commands for Bundle Algo. z/python -m monai.apps.auto3dseg NNIGen run_algo z============================================================================================================================================z#If NNI will run in your local env: zB1. Add the following line to the trialCommand in your NNI config:  z {result_dir}z--------------------------------------------------------------------------------------------------------------------------------------------z!If NNI will run in a remote env: z'1. Copy the algorithm_templates folder z+ to remote {remote_algorithm_templates_dir}z2. Copy the older z( to the remote machine {remote_algo_dir}zDThen add the following line to the trialCommand in your NNI config: z@ {remote_algo_dir} {result_dir} {remote_algorithm_templates_dir}N)loggerinfor3rrr+r1r8)rr2rrrprint_bundle_algo_instructions      z$NNIGen.print_bundle_algo_instructioncCstr tStdiS)zK Get parameter for next round of training from NNI server. ?NNI is not detected. The code will continue to run without NNI.)has_nnirZget_next_parameterrrrrrrszNNIGen.get_hyperparametersdictNoner,returncCs ||_dS)z Translate the parameter from monai bundle to meet NNI requirements. Args: params: a dict of parameters. Nr,rr,rrrr"szNNIGen.update_paramscCsddd|jDpdS) Get the identifier of the current experiment. In the format of listing the searching parameter name and values connected by underscore in the file name. r.css"|]\}}d|d|VqdS_Nr.0kvrrr sz%NNIGen.get_task_id.._Nonejoinr,itemsrrrr get_task_idszNNIGen.get_task_id.strr>rKcCs~|}tj|j}tj|||}tj|d|_t|jt rb|jj ||||ddnt |j |t|dS) Generate the record for each Algo. If it is a BundleAlgo, it will generate the config files. Args: output_folder: the directory nni will save the results to. algo_object.pklF) bundle_rootr/NrZr5r6r7r+r8rXr3r4rr;rexport_config_filer,rCrDrr>Ztask_idZ task_prefix write_pathrrrgenerates zNNIGen.generatecCstrt|ntddS)z/ Report the acc to NNI server. rFN)rGrZreport_final_resultrraccrrrr#s zNNIGen.set_scorePathLike | Noner3r>r1rKcCstj|st|dt||d\|_}|}|||||j |j |j }t t j|i}t|jfd|jji|||dSam The python interface for NNI to run. Args: obj_filename: the pickle-exported Algo object. output_folder: the root path of the algorithms templates. template_path: the algorithm_template. It must contain algo.py in the follow path: ``{algorithm_templates_dir}/{network}/scripts/algo.py`` z is not foundr0r1Nr5r6isfile ValueErrorr r+rr"retrainr, get_scorer\rSCOREr r1r#rr3r>r1algo_meta_datar,rgrrrr$s    zNNIGen.run_algo)NN)r[)r[N) r%r&r'r(r?rArErr"rZrer#r$rrrrrBs.   c@seZdZdZd$ddddddZd d Zd d Zd dZddZd%ddddddddZ dddddZ ddZ d&ddddd Z d'ddddd!d"d#Z dS)(ru Generate algorithms for the Optuna to automate hyperparameter tuning. Please refer to NNI and Optuna (https://optuna.readthedocs.io/en/stable/) for more information. Optuna has different running scheme compared to NNI. The hyperparameter samples come from a trial object (trial.suggest...) created by Optuna, so OptunaGen needs to accept this trial object as input. Meanwhile, Optuna calls OptunaGen, thus OptunaGen.__call__() should return the accuracy. Use functools.partial to wrap OptunaGen for addition input arguments. Args: algo: an Algo object (e.g. BundleAlgo). The object must at least define two methods: get_output_path and train and supports saving to and loading from pickle files via ``algo_from_pickle`` and ``algo_to_pickle``. params: a set of parameter to override the algo if override is supported by Algo subclass. Examples:: The experiment will keep generating new folders to save the model checkpoints, scripts, and configs if available. ├── algorithm_templates │ └── unet ├── unet_0 │ ├── algo_object.pkl │ ├── configs │ └── scripts ├── unet_0_learning_rate_0.01 │ ├── algo_object.pkl │ ├── configs │ ├── model_fold0 │ └── scripts └── unet_0_learning_rate_0.1 ├── algo_object.pkl ├── configs ├── model_fold0 └── scripts Notes: Different from NNI and NNIGen, OptunaGen and Optuna can be ran within the Python process. Nr)r*rI)r+r,rKcCs|d|_|dk rt|tr||dkr,||_qt||_tj|d}tj |}| ddi|jj ||f|n||_t |j|jj d|_dSr-)r3r4rr+rr5r6r7r8r9r:r;r r1r<rrrr?s  zOptunaGen.__init__cCs|jS)z(Return the dumped pickle object of algo.r@rrrrrA-szOptunaGen.get_obj_filenamecCs2tr"tdd|jdddiStdiSdS)z Get parameter for next round of training from optuna trial object. This function requires user rewrite during usage for different search space. z2Please rewrite this code by creating a child classZ learning_rateg-C6?g?zEOptuna is not detected. The code will continue to run without Optuna.N) has_optunarCrDtrialZ suggest_floatrrrrrr1s  zOptunaGen.get_hyperparameterscCs ||_dS)zSet the accuracy scoreN)rgrfrrrr#=szOptunaGen.set_scorecCs ||_dS)zSet the Optuna trialN)rt)rrtrrr set_trialAszOptunaGen.set_trialr[rr\rh)rtr3r>r1rKcCs|||||||jS)a Callable that Optuna will use to optimize the hyper-parameters Args: obj_filename: the pickle-exported Algo object. output_folder: the root path of the algorithms templates. template_path: the algorithm_template. It must contain algo.py in the follow path: ``{algorithm_templates_dir}/{network}/scripts/algo.py`` )rur$rg)rrtr3r>r1rrr__call__Es zOptunaGen.__call__rHrJcCs ||_dS)zu Translate the parameter from monai bundle. Args: params: a dict of parameters. NrLrMrrrr"UszOptunaGen.update_paramscCsddd|jDpdS)rNr.css"|]\}}d|d|VqdSrOrrQrrrrUcsz(OptunaGen.get_task_id..rVrWrrrrrZ^szOptunaGen.get_task_idr]cCs||}tj|j}tj|||}tj|d|_t|jt r`|jj |||ddnt |j |t|dS)r^r_F)r/Nrarcrrrrees zOptunaGen.generatericCstj|st|dt||d\|_}|}|||||j |j |j }t t j|i}t|jfd|jji|||dSrjrkrqrrrr$ws    zOptunaGen.run_algo)NN)r[N)r[)r[N)r%r&r'r(r?rArr#rurvr"rZrer$rrrrrs&  )' __future__rr5abcrcopyrtypingrrwarningsrmonai.apps.auto3dseg.bundle_genrmonai.apps.utilsr monai.auto3dsegr r r r monai.bundle.config_parserr monai.configr monai.utilsrmonai.utils.enumsrrrGrrsr%rC__all__rrrrrrr s(              1