a 1$eh^@sddlmZddlZddlZddlZddlmZmZddl m Z ddl m Z ddl mZddlmZddlZddlmZd Zgd Zd d Zd dZd*ddZddZddZddZddZd+ddZd,ddZd-ddZd.d d!Zd/d"d#Z d$d%Z!d0d&d'Z"d1d(d)Z#dS)2)VersionN) DataFrameSeries)import_optional_dependency)from_wkb) GeoDataFrame) _expand_user 1.0.0-beta.1)0.1.0z0.4.0r cCst|tod|vo|d S)Nz://)zhttp://zhttps://) isinstancestr startswith)urlrN/var/www/html/django/DPS/env/lib/python3.9/site-packages/geopandas/io/arrow.py_is_fsspec_url-s   rcCsR|D]D\}}t|tr$t|q|dkrt|tr|D]}|ddq:qdS)aL Older PROJ versions will not recognize IDs of datum ensemble members that were added in more recent PROJ database versions. Cf https://github.com/opengeospatial/geoparquet/discussions/110 and https://github.com/OSGeo/PROJ/pull/3221 Mimicking the patch to GDAL from https://github.com/OSGeo/gdal/pull/5872 membersidN)itemsr dict#_remove_id_from_member_of_ensembleslistpop)Z json_dictkeyvaluememberrrrr5s   rc Cs |pt}|tvr$tddti}|j|jdkD]}||}tt|j  }|ddkrd}t |dkr|d}nd}d }|j r|d kr|j }n|j }t|d d d |||i||<|j}t|r8|||d<q8|j||ptdtjddS)a*Create and encode geo metadata dict. Parameters ---------- df : GeoDataFrame schema_version : {'0.1.0', '0.4.0', '1.0.0-beta.1', None} GeoParquet specification version; if not provided will default to latest supported version. Returns ------- dict zschema_version must be one of: z, geometryr0Z geometry_typergeometry_typesNr encodingWKBcrsbbox geopandas)Zlibraryversion)primary_columncolumnsr%Zcreator)METADATA_VERSIONSUPPORTED_VERSIONS ValueErrorjoinr'dtypessortedrZ geom_typeuniqueZdropnalenr"Zto_wktZ to_json_dictrZ total_boundstolistnpisfiniteallZ_geometry_column_namer$ __version__) dfschema_versioncolumn_metadatacolseriesrZgeometry_types_namer"r#rrr_create_metadataGs@       r:cCst|dS)zEncode metadata dict to UTF-8 JSON string Parameters ---------- metadata : dict Returns ------- UTF-8 encoded JSON string utf-8)jsondumpsencode)metadatarrr_encode_metadatas r@cCs|dur dSt|dS)zDecode a UTF-8 encoded JSON string to dict Parameters ---------- metadata_str : string (UTF-8 encoded) Returns ------- dict Nr;)r<loadsdecode)Z metadata_strrrr_decode_metadatas rCcCsLt|tstd|jjdvr&tdtdd|jjD}|sHtddS)zValidate that the GeoDataFrame conforms to requirements for writing to Parquet format. Raises `ValueError` if the GeoDataFrame is not valid. copied from `pandas.io.parquet` Parameters ---------- df : GeoDataFrame z;Writing to Parquet/Feather only supports IO with DataFrames>unicodeemptystringz7Writing to Parquet/Feather requires string column namescss |]}|durt|tVqdS)N)r r ).0namerrr sz&_validate_dataframe..z!Index level names must be stringsN)r rr*r'Z inferred_typer3indexnames)r5Z valid_namesrrr_validate_dataframes  rLcCs|s td|d|d}|s*tdd}|D]$}||ddur2tdj|dq2t|d tsntd d }|d D]h\}}|D]}||vrtd j||d q|ddkrtd|dddkr~tjd|dtddq~dS)zValidate geo metadata. Must not be empty, and must contain the structure specified above. Raises ValueError if metadata is not valid. Parameters ---------- metadata : dict 9Missing or malformed geo metadata in Parquet/Feather filer%r6zI'geo' metadata in Parquet/Feather file is missing required key: 'version')r&r'NzG'geo' metadata in Parquet/Feather file is missing required key: '{key}')rr'z*'columns' in 'geo' metadata must be a dict)r zY'geo' metadata in Parquet/Feather file is missing required key '{key}' for column '{col}')rr8r r!z'Only WKB geometry encoding is supportededgesZplanarZ sphericalz'The geo metadata indicate that column 'z' has spherical edges, but because GeoPandas currently does not support spherical geometry, it ignores this metadata and will interpret the edges of the geometries as planar. stacklevel) r*getformatr rrwarningswarn UserWarning)r?r% required_keysrZrequired_col_keysr8r7rrr_validate_metadatasD   rXc Csddlm}t|t||d}i}tjr4ddi}n:|j|jdkD](}||}|j rDt j ddd qnqD|j fi|}|j ||d }|jj} | d t|i|| S) zL Helper function with main, shared logic for to_parquet/to_feather. r)Table)r6ZflavorZisorzThe GeoDataFrame contains 3D geometries, and when using shapely < 2.0, such geometries will be written not exactly following to the GeoParquet spec (not using ISO WKB). For most use cases this should not be a problem (GeoPandas can read such files fine).rP)Zpreserve_indexgeo)pyarrowrYrLr:compatZUSE_SHAPELY_20r'r,Zhas_zanyrTrUZto_wkbZ from_pandasschemar?updater@Zreplace_schema_metadata) r5rJr6rYZ geo_metadatakwargsr8r9tabler?rrr_geopandas_to_arrows&    rcsnappycKstddd}|rRd|vrR|ddurR|durR|dtvrRtjdtdd|d}t|}t|||d }|j||fd |i|dS) a Write a GeoDataFrame to the Parquet format. Any geometry columns present are serialized to WKB format in the file. Requires 'pyarrow'. This is tracking version 1.0.0-beta.1 of the GeoParquet specification at: https://github.com/opengeospatial/geoparquet. Writing older versions is supported using the `schema_version` keyword. .. versionadded:: 0.8 Parameters ---------- path : str, path object index : bool, default None If ``True``, always include the dataframe's index(es) as columns in the file output. If ``False``, the index(es) will not be written to the file. If ``None``, the index(ex) will be included as columns in the file output except `RangeIndex` which is stored as metadata only. compression : {'snappy', 'gzip', 'brotli', None}, default 'snappy' Name of the compression to use. Use ``None`` for no compression. schema_version : {'0.1.0', '0.4.0', '1.0.0-beta.1', None} GeoParquet specification version; if not provided will default to latest supported version. **kwargs Additional keyword arguments passed to pyarrow.parquet.write_table(). pyarrow.parquet(pyarrow is required for Parquet support.extrar%Nzthe `version` parameter has been replaced with `schema_version`. `version` will instead be passed directly to the underlying parquet writer unless `version` is 0.1.0 or 0.4.0.rZrPrJr6 compression) rr)rTrU FutureWarningrr rcZ write_table)r5pathrJrjr6raparquetrbrrr _to_parquet s! rnc Kstddd}ddl}t|jtdkr.td|rtd|vrt|ddurt|durt|dtvrttjd td d | d}t |}t |||d }|j ||fd |i|dS)a Write a GeoDataFrame to the Feather format. Any geometry columns present are serialized to WKB format in the file. Requires 'pyarrow' >= 0.17. This is tracking version 1.0.0-beta.1 of the GeoParquet specification for the metadata at: https://github.com/opengeospatial/geoparquet. Writing older versions is supported using the `schema_version` keyword. .. versionadded:: 0.8 Parameters ---------- path : str, path object index : bool, default None If ``True``, always include the dataframe's index(es) as columns in the file output. If ``False``, the index(es) will not be written to the file. If ``None``, the index(ex) will be included as columns in the file output except `RangeIndex` which is stored as metadata only. compression : {'zstd', 'lz4', 'uncompressed'}, optional Name of the compression to use. Use ``"uncompressed"`` for no compression. By default uses LZ4 if available, otherwise uncompressed. schema_version : {'0.1.0', '0.4.0', '1.0.0-beta.1', None} GeoParquet specification version for the metadata; if not provided will default to latest supported version. kwargs Additional keyword arguments passed to pyarrow.feather.write_feather(). pyarrow.feather(pyarrow is required for Feather support.rgrN0.17.0,pyarrow >= 0.17 required for Feather supportr%zthe `version` parameter has been replaced with `schema_version`. `version` will instead be passed directly to the underlying feather writer unless `version` is 0.1.0 or 0.4.0.rZrPrirj) rr\rr4 ImportErrorr)rTrUrkrr rcZ write_feather) r5rlrJrjr6rafeatherr\rbrrr _to_featherUs"  ruc Cs&|}|p|jj}|dus$d|vr,tdzt|dd}Wn"ttjj fybtdYn0t ||j |d}t |std|d}t |r||vr|d }t |d krtjd d d |D]N}|d|}d|vr|d}t|trt|nd}t||j|d||<qt||dS)zP Helper function with main, shared logic for read_parquet/read_feather. Nr[ziMissing geo metadata in Parquet/Feather file. Use pandas.read_parquet/read_feather() instead.rMr'zNo geometry columns are included in the columns read from the Parquet/Feather file. To read this file without geometry columns, use pandas.read_parquet/read_feather() instead.r&rrzMultiple non-primary geometry columns read from Parquet/Feather file. The first column read was promoted to the primary geometry.rPr"z OGC:CRS84)r")r)Z to_pandasr_r?r*rCrR TypeErrorr<decoderJSONDecodeErrorrXr' intersectionr/rTrUr rrrvaluesr)rbr?r5Zgeometry_columnsrr8Z col_metadatar"rrr_arrow_to_geopandassB     r}cCsddl}t|trf|durf|durft|jtdkrfzddlm}||\}}WntydYn0t |r|durt ddd}|j j |fi|pi\}}|dur|rt d|||fS) z Get the filesystem and path for a given filesystem and path. If the filesystem is not None then it's just returned as is. rNz5.0.0) FileSystemfsspecz(fsspec is requred for 'storage_options'.rgz:Cannot provide 'storage_options' with non-fsspec path '{}')r\r r rr4Z pyarrow.fsr~Zfrom_uri ExceptionrrcoreZ url_to_fsr*rS)rl filesystemstorage_optionsr\r~rrrr_get_filesystem_paths0   rcCs\ddlm}t||jr|Sz ddl}Wnty:Yn0t||jrX|||S|S)z Simplified version of pyarrow.fs._ensure_filesystem. This is only needed below because `pyarrow.parquet.read_metadata` does not yet accept a filesystem keyword (https://issues.apache.org/jira/browse/ARROW-16719) r)fsN) r\rr r~rrsZAbstractFileSystemZ PyFileSystemZ FSSpecHandler)rrrrrr_ensure_arrow_fss     rc Kstddd}|dd}t|||d\}}t|}d|d<|j|f||d |}d}|jjdusnd |jjvrzX|durt|}||} | | j}Wdq1s0Yn | |j}Wnt yYn0t ||S) a Load a Parquet object from the file path, returning a GeoDataFrame. You can read a subset of columns in the file using the ``columns`` parameter. However, the structure of the returned GeoDataFrame will depend on which columns you read: * if no geometry columns are read, this will raise a ``ValueError`` - you should use the pandas `read_parquet` method instead. * if the primary geometry column saved to this file is not included in columns, the first available geometry column will be set as the geometry column of the returned GeoDataFrame. Supports versions 0.1.0, 0.4.0 and 1.0.0-beta.1 of the GeoParquet specification at: https://github.com/opengeospatial/geoparquet If 'crs' key is not present in the GeoParquet metadata associated with the Parquet object, it will default to "OGC:CRS84" according to the specification. Requires 'pyarrow'. .. versionadded:: 0.8 Parameters ---------- path : str, path object columns : list-like of strings, default=None If not None, only these columns will be read from the file. If the primary geometry column is not included, the first secondary geometry read from the file will be set as the geometry column of the returned GeoDataFrame. If no geometry columns are present, a ``ValueError`` will be raised. storage_options : dict, optional Extra options that make sense for a particular storage connection, e.g. host, port, username, password, etc. For HTTP(S) URLs the key-value pairs are forwarded to urllib as header options. For other URLs (e.g. starting with "s3://", and "gcs://") the key-value pairs are forwarded to fsspec. Please see fsspec and urllib for more details. When no storage options are provided and a filesystem is implemented by both ``pyarrow.fs`` and ``fsspec`` (e.g. "s3://") then the ``pyarrow.fs`` filesystem is preferred. Provide the instantiated fsspec filesystem using the ``filesystem`` keyword if you wish to use its implementation. **kwargs Any additional kwargs passed to pyarrow.parquet.read_table(). Returns ------- GeoDataFrame Examples -------- >>> df = geopandas.read_parquet("data.parquet") # doctest: +SKIP Specifying columns to read: >>> df = geopandas.read_parquet( ... "data.parquet", ... columns=["geometry", "pop_est"] ... ) # doctest: +SKIP rerfrgrN)rrTZuse_pandas_metadata)r'rr[) rrrr read_tabler_r?rZopen_input_fileZ read_metadatarr}) rlr'rrarmrrbr?Z pa_filesystemsourcerrr _read_parquet s*?   , rcKsTtddd}ddl}t|jtdkr.tdt|}|j|fd|i|}t|S) a Load a Feather object from the file path, returning a GeoDataFrame. You can read a subset of columns in the file using the ``columns`` parameter. However, the structure of the returned GeoDataFrame will depend on which columns you read: * if no geometry columns are read, this will raise a ``ValueError`` - you should use the pandas `read_feather` method instead. * if the primary geometry column saved to this file is not included in columns, the first available geometry column will be set as the geometry column of the returned GeoDataFrame. Supports versions 0.1.0, 0.4.0 and 1.0.0-beta.1 of the GeoParquet specification at: https://github.com/opengeospatial/geoparquet If 'crs' key is not present in the Feather metadata associated with the Parquet object, it will default to "OGC:CRS84" according to the specification. Requires 'pyarrow' >= 0.17. .. versionadded:: 0.8 Parameters ---------- path : str, path object columns : list-like of strings, default=None If not None, only these columns will be read from the file. If the primary geometry column is not included, the first secondary geometry read from the file will be set as the geometry column of the returned GeoDataFrame. If no geometry columns are present, a ``ValueError`` will be raised. **kwargs Any additional kwargs passed to pyarrow.feather.read_table(). Returns ------- GeoDataFrame Examples -------- >>> df = geopandas.read_feather("data.feather") # doctest: +SKIP Specifying columns to read: >>> df = geopandas.read_feather( ... "data.feather", ... columns=["geometry", "pop_est"] ... ) # doctest: +SKIP rorprgrNrqrrr')rr\rr4rsr rr})rlr'rartr\rbrrr _read_featherms4r)N)NN)NrdN)NNN)N)NN)NN)N)$Zpackaging.versionrr<rTnumpyr1ZpandasrrZgeopandas._compat_compatr]rZgeopandas.arrayrr$rfiler r(r)rrr:r@rCrLrXrcrnrur}rrrrrrrrs8       =: ( 5 9 A & a