o & i=@s$dZddlmZddlZddlmZddlmZddlm Z ddl m Z ddl m Z mZmZgdZGd d d e Zd d Zed ddZedddZedddZedddZedddZedddZedddZeddod!d"Zed#e jed$dpd&d'Zed(dqd*d+Zed,d-d.Z ed/e j!ed0e j"ed1e j#ed2e j$ed3e j%j&ed4e j%j'ed5e j%j(ed6e j%j)ed7e j%j*ed8e j%j+ed9e j%j,ed:e j%j-ed;e j%j.ede j%j1ed?d@dAZ2edBdCdDZ3edEdFdGZ4edHdIdJZ5edKdrdMdNZ6edOdsdQdRZ7edSdtdUdVZ8edWdudYdZZ9ed[dvd]d^Z:ed_dwdadbZ;edcdxdedfZdS){a* Defines factories for creating layers in generic, extensible, and dimensionally independent ways. A separate factory object is created for each type of layer, and factory functions keyed to names are added to these objects. Whenever a layer is requested the factory name and any necessary arguments are passed to the factory object. The return value is typically a type but can be any callable producing a layer object. The factory objects contain functions keyed to names converted to upper case, these names can be referred to as members of the factory so that they can function as constant identifiers. eg. instance normalization is named `Norm.INSTANCE`. For example, to get a transpose convolution layer the name is needed and then a dimension argument is provided which is passed to the factory function: .. code-block:: python dimension = 3 name = Conv.CONVTRANS conv = Conv[name, dimension] This allows the `dimension` value to be set in the constructor, for example so that the dimensionality of a network is parameterizable. Not all factories require arguments after the name, the caller must be aware which are required. Defining new factories involves creating the object then associating it with factory functions: .. code-block:: python fact = LayerFactory() @fact.factory_function('test') def make_something(x, y): # do something with x and y to choose which layer type to return return SomeLayerType ... # request object from factory TEST with 1 and 2 as values for x and y layer = fact[fact.TEST, 1, 2] Typically the caller of a factory would know what arguments to pass (ie. the dimensionality of the requested type) but can be parameterized with the factory name and the arguments to pass to the created type at instantiation time: .. code-block:: python def use_factory(fact_args): fact_name, type_args = split_args layer_type = fact[fact_name, 1, 2] return layer_type(**type_args) ... kw_args = {'arg0':0, 'arg1':True} layer = use_factory( (fact.TEST, kwargs) ) ) annotationsN)Callable)Any)has_nvfuser_instance_norm)ComponentStorelook_up_optionoptional_import) LayerFactoryDropoutNormActConvPoolPadRelPosEmbedding split_argscsdeZdZdZdfdd Zdd ddZdd!ddZd"ddZd#ddZd$ddZ fddZ Z S)%r z Factory object for creating layers, this uses given factory functions to actually produce the types or constructing callables. These functions are referred to by name and can be added at any time. namestr descriptionreturnNonecs2t||d|d|ddd|_dS)NzLayer Factory 'z':  zU Please see :py:class:`monai.networks.layers.split_args` for additional args parsing.z The supported members are:)super__init__strip__doc__)selfrr __class__a/home/dell461/cl/sdc2/last_ska_mid/HISourceFinder-master-l/src/monai/networks/layers/factories.pyrRszLayerFactory.__init__Nfuncrdesc str | NonecCs^|p|jpd}|||||jdusJ|jt|jdkr#dndd|d7_dS)zj Add the factory function to this object under the given name, with optional description. Nz,  z``)raddupperlennames)rrr!r"rrrr add_factory_callableZs0z!LayerFactory.add_factory_callableclstypecs||dfdd |dS)z{ Adds a factory function which returns the supplied class under the given name, with optional description. NcsSNr)xr,rr hsz0LayerFactory.add_factory_class..r.r+)rrr,r"rr0r add_factory_classdszLayerFactory.add_factory_classcsdfdd }|S)zN Decorator for adding a factory function with the given name. r!rrcs||Sr.r2)r!rrrr _addos z+LayerFactory.factory_function.._addN)r!rrrr)rrr5rr4r factory_functionjszLayerFactory.factory_function factory_namercGs:t|tstdt|jdt||j}|j|S)z Get the constructor for the given factory name and arguments. Raises: TypeError: When ``factory_name`` is not a ``str``. zfactory_name must a str but is .) isinstancer TypeErrorr-__name__rr( componentsvalue)rr7args componentrrr get_constructorus  zLayerFactory.get_constructorcCs<t|r|St|tr|d}}n|^}}|j|g|RS)z Get the given name or name/arguments pair. If `args` is a callable it is assumed to be the constructor itself and is returned, otherwise it should be the factory name or a pair containing the name and arguments. r)callabler9rr@)rr>name_objrrr __getitem__s   zLayerFactory.__getitem__cs||jvr|St|S)z If `key` is a factory name, return it, otherwise behave as inherited. This allows referring to factory names as if they were constants, eg. `Fact.FOO` for a factory Fact with factory function foo. )r<r__getattribute__)rkeyrrr __getattr__s  zLayerFactory.__getattr__)rrrrrrr.)rrr!rr"r#rr)rrr,r-r"r#rr)rrrr)r7rrr)rr) r; __module__ __qualname__rrr+r3r6r@rCrF __classcell__rrrr r Ls   r cCsJt|tr |ifS|\}}t|tst|rt|ts!d}t|||fS)a Split arguments in a way to be suitable for using with the factory types. If `args` is a string it's interpreted as the type name. Args: args (str or a tuple of object name and kwarg dict): input arguments to be parsed. Raises: TypeError: When ``args`` type is not in ``Union[str, Tuple[Union[str, Callable], dict]]``. Examples:: >>> act_type, args = split_args("PRELU") >>> monai.networks.layers.Act[act_type] >>> act_type, args = split_args(("PRELU", {"num_parameters": 1, "init": 0.25})) >>> monai.networks.layers.Act[act_type](**args) PReLU(num_parameters=1) z_Layer specifiers must be single strings or pairs of the form (name/object-types, argument dict))r9rrAdictr:)r>rBZ name_argsmsgrrr rs rzDropout layersz$Factory for creating dropout layers.)rrzNormalization layersz*Factory for creating normalization layers.zActivation layersz'Factory for creating activation layers.zConvolution layersz(Factory for creating convolution layers.zPooling layersz$Factory for creating pooling layers.zPadding layersz$Factory for creating padding layers.z$Relative positional embedding layersz:Factory for creating relative positional embedding factorydropoutdimintr.type[nn.Dropout | nn.Dropout2d | nn.Dropout3d]cCtjtjtjf}||dS)z Dropout layers in 1,2,3 dimensions. Args: dim: desired dimension of the dropout layer Returns: Dropout[dim]d r%)nnr Dropout2d Dropout3drMtypesrrr dropout_factory rVZ alphadropoutinstance?type[nn.InstanceNorm1d | nn.InstanceNorm2d | nn.InstanceNorm3d]cCrP)z Instance normalization layers in 1,2,3 dimensions. Args: dim: desired dimension of the instance normalization layer Returns: InstanceNorm[dim]d r%)rQInstanceNorm1dInstanceNorm2dInstanceNorm3drTrrr instance_factoryrWr]batch6type[nn.BatchNorm1d | nn.BatchNorm2d | nn.BatchNorm3d]cCrP)z Batch normalization layers in 1,2,3 dimensions. Args: dim: desired dimension of the batch normalization layer Returns: BatchNorm[dim]d r%)rQ BatchNorm1d BatchNorm2d BatchNorm3drTrrr batch_factoryrWrcZinstance_nvfusercCs`|dkrtjtjf}td||dd||dSts(tdtjStdddd S) a `InstanceNorm3dNVFuser` is a faster version of InstanceNorm layer and implemented in `apex`. It only supports 3d tensors as the input. It also requires to use with CUDA and non-Windows OS. In this function, if the required library `apex.normalization.InstanceNorm3dNVFuser` does not exist, `nn.InstanceNorm3d` will be returned instead. This layer is based on a customized autograd function, which is not supported in TorchScript currently. Please switch to use `nn.InstanceNorm3d` if TorchScript is necessary. Please check the following link for more details about how to install `apex`: https://github.com/NVIDIA/apex#installation z4`InstanceNorm3dNVFuser` only supports 3d cases, use r%z instead.zd`apex.normalization.InstanceNorm3dNVFuser` is not installed properly, use nn.InstanceNorm3d instead.zapex.normalizationInstanceNorm3dNVFuser)rr)rQrZr[warningswarnrr\rrTrrr instance_nvfuser_factorys  rhgrouplayerZ localresponseZ syncbatchelureluZ leakyreluprelurelu6selucelugelusigmoidtanhsoftmaxZ logsoftmaxZswishcCddlm}|S)z= Swish activation layer. Returns: Swish rSwish) monai.networks.blocks.activationrwrvrrr swish_factory1 ryZmemswishcCru)z] Memory efficient swish activation layer. Returns: MemoryEfficientSwish rMemoryEfficientSwish)rxr|r{rrr memswish_factory>rzr}mishcCru)z; Mish activation layer. Returns: Mish rMish)rxrrrrr mish_factoryKrzrZgeglucCru)z= GEGLU activation layer. Returns: GEGLU rGEGLU)rxrrrrr geglu_factoryXrzrconv'type[nn.Conv1d | nn.Conv2d | nn.Conv3d]cCrP)z Convolutional layers in 1,2,3 dimensions. Args: dim: desired dimension of the convolutional layer Returns: Conv[dim]d r%)rQConv1dConv2dConv3drTrrr conv_factoryerWrZ convtransBtype[nn.ConvTranspose1d | nn.ConvTranspose2d | nn.ConvTranspose3d]cCrP)z Transposed convolutional layers in 1,2,3 dimensions. Args: dim: desired dimension of the transposed convolutional layer Returns: ConvTranspose[dim]d r%)rQConvTranspose1dConvTranspose2dConvTranspose3drTrrr convtrans_factorytrWrmax0type[nn.MaxPool1d | nn.MaxPool2d | nn.MaxPool3d]cCrP)z Max pooling layers in 1,2,3 dimensions. Args: dim: desired dimension of the max pooling layer Returns: MaxPool[dim]d r%)rQ MaxPool1d MaxPool2d MaxPool3drTrrr maxpooling_factoryrWrZ adaptivemaxHtype[nn.AdaptiveMaxPool1d | nn.AdaptiveMaxPool2d | nn.AdaptiveMaxPool3d]cCrP)z Adaptive max pooling layers in 1,2,3 dimensions. Args: dim: desired dimension of the adaptive max pooling layer Returns: AdaptiveMaxPool[dim]d r%)rQAdaptiveMaxPool1dAdaptiveMaxPool2dAdaptiveMaxPool3drTrrr adaptive_maxpooling_factoryrWravg0type[nn.AvgPool1d | nn.AvgPool2d | nn.AvgPool3d]cCrP)z Average pooling layers in 1,2,3 dimensions. Args: dim: desired dimension of the average pooling layer Returns: AvgPool[dim]d r%)rQ AvgPool1d AvgPool2d AvgPool3drTrrr avgpooling_factoryrWrZ adaptiveavgHtype[nn.AdaptiveAvgPool1d | nn.AdaptiveAvgPool2d | nn.AdaptiveAvgPool3d]cCrP)z Adaptive average pooling layers in 1,2,3 dimensions. Args: dim: desired dimension of the adaptive average pooling layer Returns: AdaptiveAvgPool[dim]d r%)rQAdaptiveAvgPool1dAdaptiveAvgPool2dAdaptiveAvgPool3drTrrr adaptive_avgpooling_factoryrWrZreplicationpadEtype[nn.ReplicationPad1d | nn.ReplicationPad2d | nn.ReplicationPad3d]cCrP)z Replication padding layers in 1,2,3 dimensions. Args: dim: desired dimension of the replication padding layer Returns: ReplicationPad[dim]d r%)rQReplicationPad1dReplicationPad2dReplicationPad3drTrrr replication_pad_factoryrWrZ constantpads 3    W #