a Sic @sdZddlmZddlmZmZddlZddlZddlZddl Z ddl Z ddl m Z m Z mZmZmZmZddlZddlZddlmZddlZddlZddlmZmZmZmZmZmZmZm Z m!Z!m"Z"ddl#m$Z$m%Z&m'Z'm(Z(m)Z)ddl*Z*ddl+Z+dd l,m-Z-m.Z.m/Z/m0Z0m1Z1m2Z2m3Z3dd l4m5Z5dd l6m7Z7dd l8m9Z9dd l:m;Z;ddlZ>m?Z?m@Z@ddlAmBZBeCe)e'e(ZDeDEdeFdZGe de-dZHejIGdddZJejIGdddeeZKdddddZLe"ddddd ZMe"dddd!d ZMd"d"dd#d ZMdd$d%d&d'ZNe"d~d)ddd*d+d,ZOe"ddddd*d-d,ZOdd/dd"d*d0d,ZOd1d2ZPd3ddd4d5ZQe9eBd6eBd7d8d9dd3ddd?d@ZRdddAdBdCZSdDdDdDdDdEdFdGdHdIdJ ZTeCeTUZVdeZdZUdZded<ded<ded<ded<d Zd ed <d S) IOArgsz? Return value of io/common.py:_get_filepath_or_buffer. str | BaseBufferfilepath_or_bufferstrencodingmoder compressionFbool should_closeN)__name__ __module__ __qualname____doc____annotations__r6r<r<L/var/www/html/django/DPS/env/lib/python3.9/site-packages/pandas/io/common.pyr.Ns r.c@sneZdZUdZded<ded<ejedZded<d Z d ed <d d ddZ dd ddZ dd dddZ dS) IOHandlesau Return value of io/common.py:get_handle Can be used as a context manager. This is used to easily close created buffers and to handle corner cases when TextIOWrapper is inserted. handle: The file handle to be used. created_handles: All file handles that are created by get_handle is_wrapped: Whether a TextIOWrapper needs to be detached. z IO[AnyStr]handlerr4)default_factoryzlist[IO[bytes] | IO[str]]created_handlesFr5 is_wrappedNonereturncCs\|jr8t|jtsJ|j|j|j|j|jD] }|q>g|_d|_dS)z Close all created buffers. Note: If a TextIOWrapper was inserted, it is flushed and detached to avoid closing the potentially user-created buffer. FN) rB isinstancer?r flushdetachrAremoveclose)selfr?r<r<r=rJps    zIOHandles.closezIOHandles[AnyStr]cCs|SNr<rKr<r<r= __enter__szIOHandles.__enter__r )argsrEcGs |dSrL)rJ)rKrOr<r<r=__exit__szIOHandles.__exit__N) r7r8r9r:r; dataclassesfieldlistrArBrJrNrPr<r<r<r=r>[s  r>objectr5)urlrEcCst|tsdSt|jtvS)z Check to see if a URL has a valid protocol. Parameters ---------- url : str or unicode Returns ------- isurl : bool If `url` has a valid protocol return True otherwise False. F)rFr1 parse_urlscheme _VALID_URLSrUr<r<r=is_urls rZr1)r0rEcCsdSrLr<r0r<r<r= _expand_usersr\cCsdSrLr<r[r<r<r=r\szstr | BaseBufferTcCst|trtj|S|S)a] Return the argument with an initial component of ~ or ~user replaced by that user's home directory. Parameters ---------- filepath_or_buffer : object to be converted if possible Returns ------- expanded_filepath_or_buffer : an expanded filepath or the input if not expandable )rFr1ospath expanduserr[r<r<r=r\s  rC)headerrEcCs|dur dSt|r2tt|}|dkr.tddSt|ddr|tt|}ttt|s^tdtdd|DrxtddSt |rt d tddS) NrzUPassing negative integer to header is invalid. For no header, use header=None insteadF) allow_setsz*header must be integer or list of integerscss|]}|dkVqdS)rNr<).0ir<r<r= z&validate_header_arg..z8cannot specify multi-index header with negative integerszPassing a bool to header is invalid. Use header=None for no header or header=int or list-like of ints to specify the row(s) making up the column names) r(rint ValueErrorr)rallmapanyr& TypeError)r`r<r<r=validate_header_args*   rl.r)r0convert_file_likerEcCsdSrLr<r0rmr<r<r=stringify_pathsrocCsdSrLr<rnr<r<r=rosFzFilePath | BaseBufferTcCs2|st|rtt|St|tjr*|}t|S)a Attempt to convert a path-like object to a string. Parameters ---------- filepath_or_buffer : object to be converted Returns ------- str_filepath_or_buffer : maybe a string version of the object Notes ----- Objects supporting the fspath protocol (python 3.6+) are coerced according to its __fspath__ method. Any other object is passed through unchanged, which includes bytes, strings, buffers, or anything else that's not even path-like. )r'rr,rFr]PathLike __fspath__r\rnr<r<r=ros    cOsddl}|jj|i|S)z` Lazy-import wrapper for stdlib urlopen, as that imports a big chunk of the stdlib. rN)urllib.requestrequesturlopen)rOkwargsurllibr<r<r=rtsrtzFilePath | BaseBuffercCs$t|to"tt|o"|d S)zR Returns true if the given URL looks like something fsspec can handle )zhttp://zhttps://)rFr1r5_RFC_3986_PATTERNmatch startswithrYr<r<r= is_fsspec_url s    rzstorage_optionscompression_optionsr0)r{r|utf-8rrr )r0r2r4r3r{rEc Cst|}t|\}}t||}|rJt|drJd|vrJtjdttdd}t||d}d|vr|dvr|d vrtj|d |t td|}d |vrd|vr|d7}t |t rDt |rD|pi}d dl }|jj||d }t|<} | jdd} | dkrddi}t| } Wdn1s(0Yt| ||d|dSt|rjt |t s^J|drv|dd}|dr|dd}td} g} z&tdd dlm}m}||tg} WntyYn0z$| j|fd|i|pi}Wn\t | yV|dur"ddi}nt|}d|d<| j|fd|i|pHi}Yn0t|||d|dS|rxt!dt |t t"t#j#frtt$|||d|dSt|dst|dsdt%|}t!|t|||d|dS)a% If the filepath_or_buffer is a url, translate and return the buffer. Otherwise passthrough. Parameters ---------- filepath_or_buffer : a url, filepath (str, py.path.local or pathlib.Path), or buffer {compression_options} .. versionchanged:: 1.4.0 Zstandard support. encoding : the encoding to use to decode bytes, default is 'utf-8' mode : str, optional {storage_options} .. versionadded:: 1.2.0 ..versionchange:: 1.2.0 Returns the dataclass IOArgs. writebzDcompression has no effect when passing a non-binary object as input.) stacklevelN)methodw)bz2xz)zutf-16zutf-32z( will not write the byte order mark for tr)headerszContent-EncodinggziprT)r0r2r4r6r3zs3a://zs3://zs3n://fsspecbotocore) ClientErrorNoCredentialsErrorr3anonz?storage_options passed with file object or non-fsspec file pathFreadz)Invalid file path or buffer object type: )&roget_compression_methodinfer_compressionhasattrwarningswarnRuntimeWarningr%dictUnicodeWarningrFr1rZrrrsRequestrtrgetrrr.rzryreplacer#Zbotocore.exceptionsrrPermissionError ImportErroropentuplergbytesmmapr\type)r0r2r4r3r{compression_methodZ fsspec_modervZreq_inforeqcontent_encodingreaderrZerr_types_to_retry_with_anonrrfile_objmsgr<r<r=_get_filepath_or_buffers"       ,         r)r^rEcCsddlm}td||S)z converts an absolute native path to a FILE URL. Parameters ---------- path : a path in native format Returns ------- a valid FILE URL r) pathname2urlzfile:)rrrr)r^rr<r<r=file_path_to_urls rtarrrziprzstd) .tar.tar.gz.tar.bz2.tar.xz.gz.bz2.zip.xzz.zstz"tuple[str | None, CompressionDict])r4rEc Csbt|trRt|}z|d}WqZtyN}ztd|WYd}~qZd}~00ni}|}||fS)a Simplifies a compression argument to a compression method string and a mapping containing additional arguments. Parameters ---------- compression : str or mapping If string, specifies the compression method. If mapping, value at key 'method' specifies compression method. Returns ------- tuple of ({compression method}, Optional[str] {compression arguments}, Dict[str, Any]) Raises ------ ValueError on mapping missing 'method' key rz.If mapping, compression must have key 'method'N)rFrrpopKeyErrorrg)r4compression_argsrerrr<r<r=rs "r)r| str | None)r0r4rEcCs|dur dS|dkrZt|dd}t|ts.dStD]\}}||r6|Sq6dS|tvrf|Sddgtt}d|d|}t |dS)a/ Get the compression method for filepath_or_buffer. If compression='infer', the inferred compression method is returned. Otherwise, the input compression method is returned unchanged, unless it's invalid, in which case an error is raised. Parameters ---------- filepath_or_buffer : str or file handle File path or object. {compression_options} .. versionchanged:: 1.4.0 Zstandard support. Returns ------- string or None Raises ------ ValueError on invalid compression specified. NinferT)rmUnrecognized compression type: z Valid compression types are ) rorFr1_extension_to_compressionitemslowerendswith_supported_compressionssortedrg)r0r4 extensionvalidrr<r<r=rs$   rz Path | strcCs&t|j}|s"td|ddS)z Check if parent directory of a file exists, raise OSError if it does not Parameters ---------- path: Path or str Path to check parent directory of z1Cannot save file into a non-existent directory: ''N)r parentis_dirOSError)r^rr<r<r=check_parent_directoryJs r)r2r4 memory_maperrorsr{zLiteral[False]zIOHandles[bytes]) path_or_bufr3r2r4ris_textrr{rEcCsdSrLr<rr3r2r4rrrr{r<r<r= get_handleXs r)r2r4rrrr{z Literal[True]zIOHandles[str]cCsdSrLr<rr<r<r=rgs z!IOHandles[str] | IOHandles[bytes]cCsdSrLr<rr<r<r=rvs rTcCs|pd}|pd}t||r*d|vr*|d7}t|t|trHt|t|||||d}|j} t| |\} }} t| t} t |j } | d}d|vr| rt t| |r^|dkr|j dd |_ n |dkrd|j vr|j d7_ |d kr2t| trtjf| |j d | } ntjf| |j d | } n|d krXtj| fd|j i| } n|dkrt| |j fi| } | jj dkrB| | | j} t| dkr| j| } n$| std|ntd| nb|dkr| d|j t| trtfd| i| } ntfd| i| } t| ts:Jd| jj vrB| | | j}t|dkr| j|d}|dusJ|} n$|std|ntd|n|dkrt| |j } nt|dkr0td}d|j vrd|j fi| i}nd|j!fi| i}|j| fd|j i|} nd|}t|t| trRJ| | nNt| tr|j"rd|j vrt| |j |j"|d d} n t| |j } | | d }|s|j d!krt| t#rt$| |j"d"} n~|r\|s|st| |j r\t%| d#r"t%| d$r"t%| d%s*t&| } t'| |j"|d d} | | t|jtpX|j( }d|j vrt%| d&st)d't*|jd(| +|j(rt|jtrJ| |jt,| | ||j d)S)*a Get file handle for given path/buffer and mode. Parameters ---------- path_or_buf : str or file handle File path or object. mode : str Mode to open path_or_buf with. encoding : str or None Encoding to use. {compression_options} .. versionchanged:: 1.0.0 May now be a dict with key 'method' as compression mode and other keys as compression options if compression mode is 'zip'. .. versionchanged:: 1.1.0 Passing compression options as keys in dict is now supported for compression modes 'gzip', 'bz2', 'zstd' and 'zip'. .. versionchanged:: 1.4.0 Zstandard support. memory_map : bool, default False See parsers._parser_params for more information. Only used by read_csv. is_text : bool, default True Whether the type of the content passed to the file/buffer is string or bytes. This is not the same as `"b" not in mode`. If a string content is passed to a binary file/buffer, a wrapper is inserted. errors : str, default 'strict' Specifies how encoding and decoding errors are to be handled. See the errors argument for :func:`open` for a full list of options. storage_options: StorageOptions = None Passed to _get_filepath_or_buffer .. versionchanged:: 1.2.0 Returns the dataclass IOHandles r}strictr)r2r4r3r{rr~rrr+r)filenamer3)fileobjr3rr3rzZero files found in ZIP file z9Multiple files found in ZIP file. Only one file per ZIP: rnamerrNz Zero files found in TAR archive zDMultiple files found in TAR archive. Only one file per TAR archive: r zstandardZdctxZcctxr)r2rnewlineFrb)r2readablewritableseekablerz1Expected file path name or file-like object, got z type)r?rArBr4)-_is_binary_modecodecslookuprFr1 lookup_errorrr0_maybe_memory_maprr4rrr3rrGzipFilerBZ2File _BytesZipFilebufferappendnamelistlenrrg setdefault _BytesTarFilegetnames extractfiler"r#ZstdDecompressorZZstdCompressorr2r _BytesIOWrapperr _IOWrapperr r6rkrreverser>)rr3r2r4rrrr{Zioargsr?handlesis_pathrZ zip_namesfilesfilerZ open_argsrrBr<r<r=rsH6                              cs8eZdZdZeddddZddfdd ZZS)_BufferedWriterz Some objects do not support multiple .write() calls (TarFile and ZipFile). This wrapper writes to the underlying buffer on close. rCrDcCsdSrLr<rMr<r<r=write_to_buffersz_BufferedWriter.write_to_buffercsd|jr dS|rL|d|j|WdqV1s@0Yn |jtdS)Nr)closedgetvalueseekrrrJsuperrM __class__r<r=rJs ( z_BufferedWriter.close)r7r8r9r:rrrJ __classcell__r<r<rr=rsrcsXeZdZdddddddfdd Zd d d d d ZddddZddddZZS)rNr~rzLiteral['r', 'a', 'w', 'x']z-ReadBuffer[bytes] | WriteBuffer[bytes] | NonerC)rr3r archive_namerEc s<t||_||_tjjf||||d||_dS)N)rr3r) r__init__rrtarfileTarFiler extend_moder)rKrr3rrrurr<r=rs z_BytesTarFile.__init__r1)r3rEcCsP|dd}|dkr|S|jdurLt|jj}|dvrL|d|dd}|S)Nrr+r)rrr:r)rrr suffix)rKr3rr<r<r=rs   z_BytesTarFile.extend_moderDcCsP|jdurdSt|j}|jdkr.|djS|jdvrJ|ddjS|jS)z If an explicit archive_name is not given, we still want the file inside the zip file not to be named something.tar, because that causes confusion (GH39465). Nrr+)rrr)rr r with_suffixrKrr<r<r=infer_filenames     z_BytesTarFile.infer_filenamecCs>|jp|pd}tj|d}t||_|j||dS)Nrr) rrrTarInforrsizeraddfile)rKrtarinfor<r<r=rs z_BytesTarFile.write_to_buffer)Nr~NN)r7r8r9rrrrrr<r<rr=rs rcsFeZdZddddddfdd Zdd d d Zdd d d ZZS)rNz1FilePath | ReadBuffer[bytes] | WriteBuffer[bytes]r1rrC)rr3rrEc sDt|dd}||_|dtjtj||fi||_dS)Nrr+r4) rrrrrzipfile ZIP_DEFLATEDZipFiler)rKrr3rrurr<r=rs   z_BytesZipFile.__init__rDcCs@t|jjtjtfrrZr\rlrortrzrrrvaluesrrrrrrrrrrrrr lru_cacher"r<r<r<r=s   0 $         ,"   3 #8$$$$7$