o  i{G@sddlmZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl mZddlmZmZddlmZmZmZddlmZddlmZmZddlmZdd lmZmZm Z e d \Z!Z"e d d \Z#Z$e d dd\Z%Z&erddl'm'Z'dZ(n e dded\Z'Z(gdZ)dZ*ej+ej,ej-ej.dZ/de*ddfdLd!d"Z0e0dZ1e)2d#dMd&d'Z3dNdOd.d/Z4d0d1Z5dPdQd5d6Z6 7  2 dRdSd;d<Z7d=d>Z8d?d@Z9 A  2 7 dTdUdEdFZ:dVdHdIZ; 7 A  2 7  dWdXdJdKZ|r>td t||Wd Sttt t fyc}zt d |d |d |d }~ww)zT Retrieve file from `url` to `filepath`, optionally showing a progress bar. c@seZdZdZd dd d ZdS)z)_download_with_progress..TqdmUpToz Provides `update_to(n)` which uses `tqdm.update(delta_n)`. Inspired by the example in https://github.com/tqdm/tqdm. Nbintbsizetsize int | Noner'rHcSs&|dur||_||||jdS)a! Args: b: number of blocks transferred so far, default: 1. bsize: size of each block (in tqdm units), default: 1. tsize: total size (in tqdm units). if None, remains unchanged. N)totalupdaten)selfrJrLrMr:r:r; update_togsz3_download_with_progress..TqdmUpTo.update_to)rIrIN)rJrKrLrKrMrNr'rH)__name__ __module__ __qualname____doc__rSr:r:r:r;TqdmUpToasrXBTirI)unit unit_scale unit_divisorminitersdesc) reporthookNzBtqdm is not installed, will not show the downloading progress bar.zDownload failed from  to .) has_tqdmrrCr rSwarningswarnrrrOSErrorr7error)rDrErFrXter:r:r;_download_with_progressZs& ricCst|dr |j}n t|dr|j}nt|}t|dr&|r&td|t|dr6|r6td|tj |}tj |sJd| tj vrQtd|tj ||}tj |}tj|}tj|}tj||g|kr{td ||S) zQSecurely verify compressed package member paths to prevent path traversal attacksfilenamerissymz#Symbolic link detected in archive: islnkzHard link detected in archive: z..z!Unsafe path detected in archive: zUnsafe path: path traversal )hasattrrjrr!rk ValueErrorrlr>r?normpathisabssplitr@joinrealpath commonpath)memberZ extract_toZ member_path full_pathZ extract_rootZ target_realr:r:r;safe_extract_member}s&      rwrval hash_typec s|durtd|d|d|ddSt|t}|dd}z(t|d tfd d d D]}||q1Wdn1sCwYWntyd}zt d |WYd}~dSd}~ww|| krxt d| ddStdt |d|d|ddS)a Verify hash signature of specified file. Args: filepath: path of source file to verify hash value. val: expected hash value of the file. hash_type: type of hash algorithm to use, default is `"md5"`. The supported hash types are `"md5"`, `"sha1"`, `"sha256"`, `"sha512"`. See also: :py:data:`monai.apps.utils.SUPPORTED_HASH_TYPES`. Nz Expected z is None, skip z check for file raTF)usedforsecurityrbcs dS)Ni)readr:fr:r;s zcheck_hash..zException in check_hash: zcheck_hash failed z Verified 'z', z: ) r7infor lowerropeniterrP Exceptionrf hexdigestrC)rErxryZactual_hash_funcZ actual_hashchunkrhr:r}r;rs,     "rr=hash_val gdown_kwargsrc Ks8|stdt|}td|dt|}|r=t|||s2t|d|d|d|dtd|dd Szt }t|t|}t |j d krnt sWtd d |vr_d |d <t j||fd| i|nKt |j dkrt|0}|} | dkrt|d} t| ||dntd| d|dd|dWd n1swYnt|||d|std|d|d|j} | rtj| d dt||Wd n1swYWn ttfyYnwtd|t|||st|d|d|d|d|d d S)a* Download file from specified URL link, support process bar and hash check. Args: url: source URL link to download file. filepath: target filepath to save the downloaded file (including the filename). If undefined, `os.path.basename(url)` will be used. hash_val: expected hash value to validate the downloaded file. if None, skip hash validation. hash_type: 'md5' or 'sha1', defaults to 'md5'. progress: whether to display a progress bar. gdown_kwargs: other args for `gdown` except for the `url`, `output` and `quiet`. these args will only be used if download from google drive. details of the args of it: https://github.com/wkentaro/gdown/blob/main/gdown/download.py Raises: RuntimeError: When the hash validation of the ``filepath`` existing file fails. RuntimeError: When a network issue or denied permission prevents the file download from ``url`` to ``filepath``. URLError: See urllib.request.urlretrieve. HTTPError: See urllib.request.urlretrieve. ContentTooShortError: See urllib.request.urlretrieve. IOError: See urllib.request.urlretrieve. RuntimeError: When the hash validation of the ``url`` downloaded file fails. razDefault downloading to ''z) check of existing file failed: filepath= , expected =z File exists: z, skipped downloading.Ndrive.google.comzITo download files from Google Drive, please install the gdown dependency.fuzzyTquietzcloud-api.yandex.nethref)rFzDownload of file from z, received from  r`z2 failed due to network issue or denied permission.exist_okz Downloaded: z& check of downloaded file failed: URL=z , filepath=)rrCresolver7rexistsr RuntimeErrortempfileTemporaryDirectoryr netloc has_gdownrdownloadr getcodejsonloadriparentr>makedirsshutilmovePermissionErrorNotADirectoryError) rDrErryrFrtmp_dirtmp_nameresponsecoderZfile_dirr:r:r;rsx#      rc Cst|dU}|D]G}t||}|rq tjtj|dd| |#}t |d}t ||Wdn1s>wYWdn1sMwYq WddS1s^wYdSNrTrwb) zipfileZipFileinfolistrwis_dirr>rr?dirnamerr copyfileobj)rE output_dirzip_fileru safe_pathsourcetargetr:r:r; _extract_zips    "rc Cst|d[}|D]M}t||}|sq tjtj|dd| |}|durX|#t|d}t ||Wdn1sDwYWdn1sSwYq WddS1sdwYdSr) tarfiler getmembersrwisfiler>rr?r extractfilerr)rErZtar_filerurrrr:r:r; _extract_tar!s"    "rrar file_typehas_basec Cs|rt|t|dd}nt|}|r+t|ddur+td|ddSt|}|rGt|||sGt |d|d|d|dtd |d| }|j d s`|d krgt||dS|j d sw|j d swd |vr~t||dStd |d|d)aK Extract file to the output directory. Expected file types are: `zip`, `tar.gz` and `tar`. Args: filepath: the file path of compressed file. output_dir: target directory to save extracted files. hash_val: expected hash value to validate the compressed file. if None, skip hash validation. hash_type: 'md5' or 'sha1', defaults to 'md5'. file_type: string of file type for decompressing. Leave it empty to infer the type from the filepath basename. has_base: whether the extracted files have a base folder. This flag is used when checking if the existing folder is a result of `extractall`, if it is, the extraction is skipped. For example, if A.zip is unzipped to folder structure `A/*.png`, this flag should be True; if B.zip is unzipped to `*.png`, this flag should be False. Raises: RuntimeError: When the hash validation of the ``filepath`` compressed file fails. NotImplementedError: When the ``filepath`` file extension is not one of [zip", "tar.gz", "tar"]. rarNzNon-empty folder exists in z, skipped extracting.z+ check of compressed file failed: filepath=rrzWriting into directory: ziptarztar.gzzMUnsupported file type, available options are: ["zip", "tar.gz", "tar"]. name=z type=)rrCrqrnextiterdirr7rrrrstriprendswithrrNotImplementedError)rErrryrr cache_dirZ _file_typer:r:r;r/s,    rdata_urlc CszNtj|dd}|jd}|rtd|}|rt|dWSd|vrJt|}d|jdd vrJt|jd }| d d d i}|rJt| djWSt |WSt yb}zt d||d}~ww)z- Get the filename from the URL link. T)allow_redirectszContent-Dispositionzfilename="?([^";]+)"?rrz text/htmlz Content-Typer=z html.parserspanclassz uc-name-sizeazError processing URL: N) rheadheadersgetrefindallr!rtextfindrCr)rrcontent_dispositionrjsoupZ filename_divrhr:r:r;get_filename_from_urlfs&     rc Csdtt|j}dtt|j} |dvr0| dkr0t||} td|d| | }| r@| |kr@td|d| t %} |pOt| t| } t || |||dt | |||dWd d S1slwYd S) ak Download file from URL and extract it to the output directory. Args: url: source URL link to download file. filepath: the file path of the downloaded compressed file. use this option to keep the directly downloaded compressed file, to avoid further repeated downloads. output_dir: target directory to save extracted files. default is the current directory. hash_val: expected hash value to validate the downloaded file. if None, skip hash validation. hash_type: 'md5' or 'sha1', defaults to 'md5'. file_type: string of file type for decompressing. Leave it empty to infer the type from url's base file name. has_base: whether the extracted files have a base folder. This flag is used when checking if the existing folder is a result of `extractall`, if it is, the extraction is skipped. For example, if A.zip is unzipped to folder structure `A/*.png`, this flag should be True; if B.zip is unzipped to `*.png`, this flag should be False. progress: whether to display progress bar. r=)r=raz filepath=z=, which missing file extension. Auto-appending extension to: z,File extension mismatch: expected extension z , but get )rDrErryrF)rErrrN)rrrrsuffixesrC with_suffixr7warningrnrrrrr) rDrErrryrrrFZurl_filename_extZ filepath_extZ new_filepathrrjr:r:r;r}s   "r) r r!r"r!r#r$r%r&r'r()r<r r'r!)T)rDr!rErrFrGr'rH)Nr)rEr rxr$ryr!r'rG)r=NrT)rDr!rEr rr$ryr!rFrGrrr'rH)raNrr=T)rEr rr rr$ryr!rr!rrGr'rH)rr!r'r!)r=raNrr=TT)rDr!rEr rr rr$ryr!rr!rrGrFrGr'rH)= __future__rhashlibrr)r>rrr2rrrcrpathlibrtypingrr urllib.errorrrr urllib.parser urllib.requestr r monai.config.type_definitionsr monai.utilsr rrr has_requestsrrrZhas_bs4rrb__all__Z DEFAULT_FMTrrrrrrr7appendrCrirwrrrrrrrr:r:r:r;s|          # $ V  7