U Ph@s^dZddlmZddlZddlZddlZddlmZddlm Z m Z m Z ddl m Z ddlmZddlmZddlZddlZddlmZdd lmZdd lmZdd lmZmZmZdd lm Z m!Z!m"Z"dd l#m$Z$m%Z%e!d\Z&Z'e!d\Z(Z'e!d\Z)Z'dddddddddddddddd d!d"d#d$d%d&gZ*ee+d'Z,da-d(d&Z.d}d*d+d,d$Z/d*d+d-d%Z0ej1d.fd/d0d1d0d/d2d3dZ2d~d/d5d6d7d8d9dZ3dd:d;d5d5d/d<d=dZ4dd/d>d>d5d5d/d?d@dZ5dAejj6j7fd6dBdCdDdEdZ8ej6j9fdFdZ:d/d0d0d/dGdHdZ;e dIdJdKdZddMdMdRdSdZ?dTdUdVdWdZ@ddIdYdZdZd[d\d]d5d^d5dZd6d6d5d_d`dZAddId]dad5dbd^d6d6d5dc dddZBd>d>d>d0d*dZdZdedfdgZCddId*d>dkd5d]d5d[dldZdZd6d6dm dndZDdod ZEdpd!ZFddqd*dqdrd5d5dCdsdtduZGddqd*dqd5d5drdvdwd"ZHe ddqd*dqd5d5dxdyd#ZIddIdzd{d|ZJdS)zE Utilities and types for defining networks, these depend on PyTorch. ) annotationsN) OrderedDict)CallableMappingSequence)contextmanager)deepcopy)Any) get_logger)PathLike) ensure_tuplesave_objset_determinism)look_up_optionoptional_import pytorch_after)convert_to_dst_typeconvert_to_tensoronnxzonnx.reference onnxruntimeone_hotpredict_segmentationnormalize_transformto_norm_affine normal_init icnr_init pixelshuffle eval_mode train_modeget_state_dictcopy_model_state save_stateconvert_to_onnxconvert_to_torchscriptconvert_to_trt meshgrid_ij meshgrid_xyreplace_modulesreplace_modules_templook_up_named_moduleset_named_modulehas_nvfuser_instance_norm) module_namecCsXtdk r tStddd\}ats$dSzddl}|dWntk rRdaYnXtS)zwhether the current environment has InstanceNorm3dNVFuser https://github.com/NVIDIA/apex/blob/23.05-devel/apex/normalization/instance_norm.py#L15-L16 Nzapex.normalizationZInstanceNorm3dNVFusernameFrZinstance_norm_nvfuser_cuda) _has_nvfuserr importlib import_module ImportError)_r0r4I/home/dell461/cl/sdc2/HISourceFinder-master-l/src/monai/networks/utils.pyr+Fs Fstrr-cCst|dd|Dd|d}|dkr*dS|dkr6|S|dD]P}|rZ|t|}q@t|dd|Dddd}|dkrdSt||}q@|S) a get the named module in `mod` by the attribute name, for example ``look_up_named_module(net, "features.3.1.attn")`` Args: name: a string representing the module attribute. mod: a pytorch module to be searched (in ``mod.named_modules()``). print_all_options: whether to print all named modules when `name` is not found in `mod`. Defaults to False. Returns: the corresponding pytorch module's subcomponent such as ``net.features[3][1].attn`` cSsh|] }|dqSrr4.0nr4r4r5 hsz'look_up_named_module..N)defaultprint_all_options.cSsh|] }|dqSr7r4)r9itemr4r4r5r;rsF)r named_modulessplitisdigitintgetattr)r.modr=name_strr:r4r4r5r)Zs$  cCsJ|dd}t|dkr|nd|f\}}|s0|St||}t||||S)a look up `name` in `mod` and replace the layer with `new_layer`, return the updated `mod`. Args: mod: a pytorch module to be updated. name: a string representing the target module attribute. new_layer: a new module replacing the corresponding layer at ``mod.name``. Returns: an updated ``mod`` See also: :py:func:`monai.networks.utils.look_up_named_module`. r?r>)rsplitlenr)setattr)rFr.Z new_layerZ mods_attrsubmodsattr_modr4r4r5r*ys   rHz torch.TensorrDz torch.dtype)labels num_classesdtypedimreturncCs|j|dkr=1 Returns: Reshuffled version of `x`. Raises: ValueError: When input channels of `x` are not divisible by (scale_factor ** spatial_dims) NrIrzNumber of input channels (z:) must be evenly divisible by scale_factor ** dimensions (z**=z).csg|] }|qSr4r4)r9dfactorr4r5 xsz pixelshuffle..rH)rZrUrrDrangeextendr]permute) rrrrS input_size batch_sizechannelsZ scale_divisorZ org_channels output_sizeindicesZpermute_indicesidxr4rr5rXs$    " $z nn.Module)netsc gsZdd|D}z(tdd|DVW5QRXW5|D]}t|dr<|q.traincSs"g|]}t|dr|n|qS)eval)rrr8r4r4r5rsN)rrr\no_grad)rrr:r4r4r5rs  c gs\dd|D}z*tddd|DVW5QRXW5|D]}t|dr>|q>XdS)a Set network(s) to train mode and then return to original state at the end. Args: nets: Input network(s) Examples .. code-block:: python t=torch.rand(1,1,16,16) p=torch.nn.Conv2d(1,1,3) p.eval() print(p.training) # False with train_mode(p): print(p.training) # True print(p(t).sum().backward()) # No exception cSs g|]}t|dr|js|qSrrr8r4r4r5rs ztrain_mode..rTcSs"g|]}t|dr|n|qS)r)rrr8r4r4r5rsN)rrr\set_grad_enabled)rZ eval_listr:r4r4r5rs  ztorch.nn.Module | MappingobjcCs0t|tjtjjfr|j}t|dr,|S|S)z Get the state dict of input object if has `state_dict`, otherwise, return object directly. For data parallel model, automatically convert it to regular model first. Args: obj: input object to check and get the state_dict. state_dict)rr DataParallelparallelDistributedDataParallelmodulerrrr4r4r5rs r>T)rsrccst|}tt|}fdd|D} t|t} } |D]F\} } || }||kr>|| kr>||j| jkr>| ||<| |q>|r|niD]r} ||| }||kr|| kr||j|| jkrtd||jd|| jd|| ||<| |q|dk rd|D]J\}}|||}|dk r|d| kr|d||d<| |dqtt | } tt |  | }t d t | d t |d |rt|tjjrt|tjtjjfr|j}|||| |fS) a Compute a module state_dict, of which the keys are the same as `dst`. The values of `dst` are overwritten by the ones from `src` whenever their keys match. The method provides additional `dst_prefix` for the `dst` key when matching them. `mapping` can be a `{"src_key": "dst_key"}` dict, indicating `dst[dst_prefix + dst_key] = src[src_key]`. This function is mainly to return a model state dict for loading the `src` model state into the `dst` model, `src` and `dst` can have different dict keys, but their corresponding values normally have the same shape. Args: dst: a pytorch module or state dict to be updated. src: a pytorch module or state dict used to get the values used for the update. dst_prefix: `dst` key prefix, so that `dst[dst_prefix + src_key]` will be assigned to the value of `src[src_key]`. mapping: a `{"src_key": "dst_key"}` dict, indicating that `dst[dst_prefix + dst_key]` to be assigned to the value of `src[src_key]`. exclude_vars: a regular expression to match the `dst` variable names, so that their values are not overwritten by `src`. inplace: whether to set the `dst` module with the updated `state_dict` via `load_state_dict`. This option is only available when `dst` is a `torch.nn.Module`. filter_func: a filter function used to filter the weights to be loaded. See 'filter_swinunetr' in "monai.networks.nets.swin_unetr.py". Examples: .. code-block:: python from monai.networks.nets import BasicUNet from monai.networks.utils import copy_model_state model_a = BasicUNet(in_channels=1, out_channels=4) model_b = BasicUNet(in_channels=1, out_channels=2) model_a_b, changed, unchanged = copy_model_state( model_a, model_b, exclude_vars="conv_0.conv_0", inplace=False) # dst model updated: 76 of 82 variables. model_a.load_state_dict(model_a_b) # Returns: an OrderedDict of the updated `dst` state, the changed, and unchanged keys. cs$h|]}rt|r|qSr4recompilesearchr9Zs_key exclude_varsr4r5r;sz#copy_model_state..zParam. shape changed from z to r?NrrHz'dst' model updated:  of z variables.)rrrZitemsr[appendrkrlsortedset differenceloggerinforKrr\rModulerrrrload_state_dict)rrZ dst_prefixmappingrinplace filter_funcsrc_dictZdst_dictZto_skipall_keysZ updated_keyssvalZdst_keykeyrXZnew_pairZunchanged_keysr4rr5r s<1    $       ztorch.nn.Module | dictr )rpathcKsNi}t|tr.|D]\}}t|||<qnt|}tf||d|dS)a Save the state dict of input source data with PyTorch `save`. It can save `nn.Module`, `state_dict`, a dictionary of `nn.Module` or `state_dict`. And automatically convert the data parallel module to regular module. For example:: save_state(net, path) save_state(net.state_dict(), path) save_state({"net": net, "opt": opt}, path) net_dp = torch.nn.DataParallel(net) save_state(net_dp, path) Refer to: https://pytorch.org/ignite/v0.4.8/generated/ignite.handlers.DiskSaver.html. Args: src: input data to save, can be `nn.Module`, `state_dict`, a dictionary of `nn.Module` or `state_dict`. path: target file path to save the input object. kwargs: other args for the `save_obj` except for the `obj` and `path`. default `func` is `torch.save()`, details of the args: https://pytorch.org/docs/stable/generated/torch.save.html. )rrN)rdictrrr )rrkwargsZckptkvr4r4r5r!/s  -C6?z Sequence[Any]zSequence[str] | Nonez int | NonezDMapping[str, Mapping[int, str]] | Mapping[str, Sequence[int]] | Nonez Any | Noneztorch.device | None)modelinputs input_names output_names opset_version dynamic_axesfilenameverifyrVuse_ort ort_providerrtolatol use_tracec s4|ti}| r |}n= 7 and not exceed the latest opset version supported by PyTorch, for more details: https://github.com/onnx/onnx/blob/main/docs/Operators.md and https://github.com/pytorch/pytorch/blob/master/torch/onnx/_constants.py dynamic_axes: specifies axes of tensors as dynamic (i.e. known only at run-time). If set to None, the exported model will have the shapes of all input and output tensors set to match given ones, for more details: https://pytorch.org/docs/stable/onnx.html#torch.onnx.export. filename: optional filename to save the ONNX model, if None, don't save the ONNX model. verify: whether to verify the ONNX model with ONNX or onnxruntime. device: target PyTorch device to verify the model, if None, use CUDA if available. use_ort: whether to use onnxruntime to verify the model. ort_provider": onnxruntime provider to use, default is ["CPUExecutionProvider"]. rtol: the relative tolerance when comparing the outputs of PyTorch model and TorchScript model. atol: the absolute tolerance when comparing the outputs of PyTorch model and TorchScript model. use_trace: whether to use `torch.jit.trace` to export the torchscript model. kwargs: other arguments except `obj` for `torch.jit.script()` to convert model, for more details: https://pytorch.org/docs/master/generated/torch.jit.script.html. rH example_outputszexample_outputs is required in scripting mode before PyTorch 1.10.Please provide example outputs or use trace mode to export onnx model.N)frrrrcudarcs&g|]}t|tjr|n|qSr4rr\rrxr9irVr4r5rsz#convert_to_onnx..rseedTcSsg|] }|jqSr4r-rr4r4r5rscSsg|]}|qSr4)rrrr4r4r5rsZCPUExecutionProvider)Z providers rtrr)'rr\rrrjitscriptioBytesIOrexportrZload_model_from_stringgetvalueloadrVr is_availablerxrr graphinputrziprZInferenceSessionSerializeToStringrun onnxreferenceZReferenceEvaluatorrrtesting assert_closeassert_allcloserrrR)rrrrrrrrrVrrrrrrZtorch_versioned_kwargsZmode_to_exportr onnx_model torch_outZmodel_input_namesZ input_dictZort_sessZonnx_outZsessr1r2 assert_fnr4rr5r"Qs-             "z dict | NonezSequence[Any] | None) rfilename_or_obj extra_filesrrrVrrrc  s|t^|r@|dkr&tdtjj|fd|i| } ntjj|f| } |dk rjtjj| ||dW5QRX|rdkrttj rdnd|dkrtdfdd |D}|dk rtj |n| } | | }t<t d d t||} t d d t| |} t dd W5QRXt| | D]R\}}t|tjsdt|tjr@td d rxtjjntjj}|||||dq@| S)a Utility to convert a model into TorchScript model and save to file, with optional input / output data verification. Args: model: source PyTorch model to save. filename_or_obj: if not None, specify a file-like object (has to implement write and flush) or a string containing a file path name to save the TorchScript model. extra_files: map from filename to contents which will be stored as part of the save model file. for more details: https://pytorch.org/docs/stable/generated/torch.jit.save.html. verify: whether to verify the input and output of TorchScript model. if `filename_or_obj` is not None, load the saved TorchScript model and verify. inputs: input test data to verify model, should be a sequence of data, every item maps to a argument of `model()` function. device: target device to verify the model, if None, use CUDA if available. rtol: the relative tolerance when comparing the outputs of PyTorch model and TorchScript model. atol: the absolute tolerance when comparing the outputs of PyTorch model and TorchScript model. use_trace: whether to use `torch.jit.trace` to export the TorchScript model. kwargs: other arguments except `obj` for `torch.jit.script()` or `torch.jit.trace()` (if use_trace is True) to convert model, for more details: https://pytorch.org/docs/master/generated/torch.jit.script.html. Nz'Missing input data for tracing convert.example_inputs)rr _extra_filesrr$Missing input data for verification.cs&g|]}t|tjr|n|qSr4rrrr4r5rsz*convert_to_torchscript..rrrHrr)rr\rrrtracersaverVrr r rxrr rrrrrrr)rrrrrrVrrrr script_moduleZts_modelrZtorchscript_outrrrr4rr5r#s:"        min_shape opt_shape max_shaperV precisionrrcCs\tdd\}} tdd\} } |||f} |s.gn|}|s:gn|}| |||jj} || } | dt|jj>}| }|r|j |df| | || }| | }|sd}t|jD]}|||d7}qtd || }|||d kr||jj| ||}t}||| jj|t d |||d }|S) af This function takes an ONNX model as input, exports it to a TensorRT engine, wraps the TensorRT engine to a TensorRT engine-based TorchScript model and return the TorchScript model. Args: onnx_model: the source ONNX model to compile. min_shape: the minimum input shape of the converted TensorRT model. opt_shape: the optimization input shape of the model, on which the TensorRT optimizes. max_shape: the maximum input shape of the converted TensorRT model. device: the target GPU index to convert and verify the model. precision: the weight precision of the converted TensorRT engine-based TorchScript model. Should be 'fp32' or 'fp16'. input_names: optional input names of the ONNX model. Should be a sequence like `['input_0', 'input_1', ..., 'input_N']` where N equals to the number of the model inputs. output_names: optional output names of the ONNX model. Should be a sequence like `['output_0', 'output_1', ..., 'output_N']` where N equals to the number of the model outputs. Ztensorrtz8.5.3torch_tensorrt1.4.0rHrr> z.TensorRT cannot parse the ONNX model, due to: Zfp16cuda:)rVZinput_binding_namesZoutput_binding_names)!r set_deviceLoggerWARNINGBuilderZcreate_networkrDZNetworkDefinitionCreationFlagZEXPLICIT_BATCHZcreate_optimization_profile set_shapeZ OnnxParserparserrZ num_errorsZ get_errordesc ExceptionZcreate_builder_configZadd_optimization_profileZset_flagZ BuilderFlagZFP16Zbuild_serialized_networkrrwritetsZembed_engine_in_new_moduler r\rV)rr#r$r%rVr&rrZtrtr3r' input_shapesrbuildernetworkprofileparsersuccessZparser_error_messagerconfigZserialized_enginer trt_modelr4r4r5_onnx_trt_compilesB          r=Zinput_0Zoutput_0{Gz?zSequence[int] | Nonez bool | None) rr& input_shapedynamic_batchsizerrrrVuse_onnxonnx_input_namesonnx_output_namesrrc Kstddd\}}tjs"td|s.td|sDtd|d|dk rjt|d krjtd |d |rr|nd }|rt d |nt d}|dkrtj ntj }t t ||g}ddddd}|r|||d }|||d}|||d}n |}}}||}t||||d}||r| rJdd| Dni}|| rfdd| Dnit||| | ||d}t||||||| | d}nf||tNtjj |d4|j|||dg}|j|f|||dd | }W5QRXW5QRX|r|dkrtd!|dk r.tj|n|}t<td d"t ||}td d"t ||}tdd"W5QRXt||D]R\}}t|tjst|tjrtdd#rtjjntjj }|||| | d$q|S)%a Utility to export a model into a TensorRT engine-based TorchScript model with optional input / output data verification. There are two ways to export a model: 1, Torch-TensorRT way: PyTorch module ---> TorchScript module ---> TensorRT engine-based TorchScript. 2, ONNX-TensorRT way: PyTorch module ---> TorchScript module ---> ONNX model ---> TensorRT engine ---> TensorRT engine-based TorchScript. When exporting through the first way, some models suffer from the slowdown problem, since Torch-TensorRT may only convert a little part of the PyTorch model to the TensorRT engine. However when exporting through the second way, some Python data structures like `dict` are not supported. And some TorchScript models are not supported by the ONNX if exported through `torch.jit.script`. Args: model: a source PyTorch model to convert. precision: the weight precision of the converted TensorRT engine based TorchScript models. Should be 'fp32' or 'fp16'. input_shape: the input shape that is used to convert the model. Should be a list like [N, C, H, W] or [N, C, H, W, D]. dynamic_batchsize: a sequence with three elements to define the batch size range of the input for the model to be converted. Should be a sequence like [MIN_BATCH, OPT_BATCH, MAX_BATCH]. After converted, the batchsize of model input should between `MIN_BATCH` and `MAX_BATCH` and the `OPT_BATCH` is the best performance batchsize that the TensorRT tries to fit. The `OPT_BATCH` should be the most frequently used input batchsize in the application, default to None. use_trace: whether using `torch.jit.trace` to convert the PyTorch model to a TorchScript model and then convert to a TensorRT engine based TorchScript model or an ONNX model (if `use_onnx` is True), default to False. filename_or_obj: if not None, specify a file-like object (has to implement write and flush) or a string containing a file path name to load the TensorRT engine based TorchScript model for verifying. verify: whether to verify the input and output of the TensorRT engine based TorchScript model. device: the target GPU index to convert and verify the model. If None, use #0 GPU. use_onnx: whether to use the ONNX-TensorRT way to export the TensorRT engine-based TorchScript model. onnx_input_names: optional input names of the ONNX model. This arg is only useful when `use_onnx` is True. Should be a sequence like `('input_0', 'input_1', ..., 'input_N')` where N equals to the number of the model inputs. If not given, will use `('input_0',)`, which supposes the model only has one input. onnx_output_names: optional output names of the ONNX model. This arg is only useful when `use_onnx` is True. Should be a sequence like `('output_0', 'output_1', ..., 'output_N')` where N equals to the number of the model outputs. If not given, will use `('output_0',)`, which supposes the model only has one output. rtol: the relative tolerance when comparing the outputs between the PyTorch model and TensorRT model. atol: the absolute tolerance when comparing the outputs between the PyTorch model and TensorRT model. kwargs: other arguments except `module`, `inputs`, `enabled_precisions` and `device` for `torch_tensorrt.compile()` to compile model, for more details: https://pytorch.org/TensorRT/py_api/torch_tensorrt.html#torch-tensorrt-py. r'r()versionzCannot find any GPU devices.z*Missing the input shape for model convert.z@There is no dynamic batch range. The converted model only takes z shape input.NrzAThe dynamic batch range sequence should have 3 elements, but got z elements.rr*zcuda:0Zfp32rrD)rA scale_numcSs|}|d|9<|S)Nrr4)rArGZ scale_shaper4r4r5scale_batch_sizesz(convert_to_trt..scale_batch_sizerHrI)rVrrcSsi|]}|ddiqSrZ batchsizer4r9rr4r4r5 sz"convert_to_trt..cSsi|]}|ddiqSrIr4rJr4r4r5rKs)rrr"r)r#r$r%Z torchscript)rZenabled_precisionsrVirrrrr)!rr\rr r2rrkrlrKrVfloat32halfrandr rxrr#updater"r=rInputrrr rrrrrrrr) rr&rArBrrrrVrCrDrErrrr'r3 target_deviceZconvert_precisionrrHZmin_input_shapeZopt_input_shapeZmax_input_shapeZir_modelrr<Zinput_placeholderrZtrt_outrrrr4r4r5r$\s:            cGs2tjjdk r(dtjjkr(tj|ddiStj|S)Nindexingijr\meshgrid__kwdefaults__tensorsr4r4r5r%scGsJtjjdk r(dtjjkr(tj|ddiStj|d|df|ddS)NrSxyrHrrIrUrXr4r4r5r&sztorch.nn.Modulez!list[tuple[str, torch.nn.Module]])parentr. new_moduleout strict_match match_devicerTc s|r4tdd|D}t|dkr4||d|d}|dkr|d|t|}||dd}g}t|||||fdd |D7}nZ|rt||} t|||||| fg7}n0|D]&\} } || krt|| t ||d d qdS) zO Helper function for :py:class:`monai.networks.utils.replace_modules`. cSsh|] }|jqSr4rrr4r4r5r;sz#_replace_modules..rHrr?rsNcs&g|]}d|d|dfqS)r?rrHr4)r9r parent_namer4r5rsz$_replace_modules..T)r^) rZ parametersrKrxrrE_replace_modulesrLrAr) r[r.r\r]r^r_devicesrZ_outZ old_modulemod_namer3r4rar5rds&       rd)r[r.r\r^r_rTcCsg}t|||||||S)a Replace sub-module(s) in a parent module. The name of the module to be replace can be nested e.g., `features.denseblock1.denselayer1.layers.relu1`. If this is the case (there are "." in the module name), then this function will recursively call itself. Args: parent: module that contains the module to be replaced name: name of module to be replaced. Can include ".". new_module: `torch.nn.Module` to be placed at position `name` inside `parent`. This will be deep copied if `strict_match == False` multiple instances are independent. strict_match: if `True`, module name must `== name`. If false then `name in named_modules()` will be used. `True` can be used to change just one module, whereas `False` can be used to replace all modules with similar name (e.g., `relu`). match_device: if `True`, the device of the new module will match the model. Requires all of `parent` to be on the same device. Returns: List of tuples of replaced modules. Element 0 is module name, element 1 is the replaced module. Raises: AttributeError: if `strict_match` is `True` and `name` is not a named module in `parent`. rd)r[r.r\r^r_r]r4r4r5r'+s )r[r.r\r^r_ccsJg}zt||||||dVW5|D]\}}t|||gd|dq&XdS)z Temporarily replace sub-module(s) in a parent module (context manager). See :py:class:`monai.networks.utils.replace_modules`. T)r^r_Nrg)r[r.r\r^r_replacedrr4r4r5r(Ps   )rc sdk rdk rtdt|}t}dk rfdd|D}|D]@\}}||krjd|_||qH|jsHd|_td|dqHdk rfd d|D}|D]@\}}||krd|_||q|jsd|_td |dqt t |d t |d dS) a  A utilty function to help freeze specific layers. Args: model: a source PyTorch model to freeze layer. freeze_vars: a regular expression to match the `model` variable names, so that their `requires_grad` will set to `False`. exclude_vars: a regular expression to match the `model` variable names, except for matched variable names, other `requires_grad` will set to `False`. Raises: ValueError: when freeze_vars and exclude_vars are both specified. NzEIncompatible values: freeze_vars and exclude_vars are both specified.cs$h|]}rt|r|qSr4rr) freeze_varsr4r5r;}sz freeze_layers..FTz!The freeze_vars does not include z0, but requires_grad is False, change it to True.cs$h|]}rt|r|qSr4rrrr4r5r;szThe exclude_vars includes rz variables frozen.) rrrZnamed_parametersr}rrkrlrrrK) rrirrZ frozen_keysZ to_freezer.paramZ to_excluder4)rrir5 freeze_layershs2   rl)F)Frd)NNFF)FF)r>NNTN) NNNNNFNFNrrdT)NNFNNrrdF) NFNFNFr>r?r@rd)TT)TT)TT)NN)K__doc__ __future__rrrrk collectionsrcollections.abcrrr contextlibrcopyrtypingr rrr\torch.nnrmonai.apps.utilsr monai.configr monai.utils.miscr r rmonai.utils.modulerrrmonai.utils.type_conversionrrrr3rr__all__rrr/r+r)r*rfrrrrrnormal_rkaiming_normal_rrrrrr r!r"r#r=r$r%r&rdr'r(rlr4r4r4r5 s            52)-"#U%*{ GO( *%