Manage your XYZ Hub or HERE Data Hub spaces from Python.

Overview

XYZ Spaces for Python

Documentation Status Tests PyPI - Status PyPI - Python Version PyPI - Implementation Downloads Conda (channel only) Conda Downloads PyPI - License LGTM alerts LGTM context Swagger Validator GitHub contributors Codecov Slack Code style: black commits since Anaconda-Server Badge Binder

Manage your XYZ Hub or HERE Data Hub spaces and Interactive Map Layer from Python.

FEATURED IN: Online Python Machine Learning Conference & GeoPython 2020, Sept 21, 2020, see conference schedule.

Motivation

XYZ is an Open Source, real-time, cloud database system providing access to large geospatial data at scale. An XYZ "Hub" manages "spaces" that contain "features" (geodata "records") with tags and properties, with spaces and features having unique IDs. A RESTful API exists to provide low-level access to interact with a XYZ Hub.

This Python package allows to interact with your XYZ spaces and features on a given Hub using a higher level programmatic interface that wraps the RESTful API. Using this package you can:

  • Create, read, list, update, share, delete spaces (also: get space info and stats).
  • Add, read, update, iterate, search, cluster (hex/quad bins), delete features.
  • Search features by ID, tag, property, bbox, tile, radius, geometry.

Based on the XYZ Hub the HERE Data Hub is a commercial service (with a free plan), that offers some additional features (in a pro plan), like clustering, virtual spaces, activity logs, and likely more to come.

The GIF below shows an interaction with an example notebook, demonstrating how to use a spatial search on a big public dataset, loaded from the HERE Data Hub.

Example from xyzspaces building_numbers.ipynb notebook

Prerequisites

Before you can install this package, run its test-suite or use the example notebooks to make sure your system meets the following prerequisities:

  • A Python installation, 3.7+ recommended, with the pip command available to install dependencies

  • A HERE developer account, free and available under HERE Developer Portal

  • An XYZ API access token from your XYZ Hub server or the XYZ portal (see also its Getting Started section) in an environment variable named XYZ_TOKEN which you can set like this (with a valid value, of course):

    export XYZ_TOKEN="MY-FANCY-XYZ-TOKEN"

    If you prefer, you can alternatively provide this token as a parameter in your code.

Installation

This package can be installed with pip or conda from various sources:

  • Install with conda from the Anaconda conda-forge channel:

    conda install -c conda-forge xyzspaces
  • Install from the Python Package Index:

    pip install xyzspaces
  • Install from the Python Package Index with optional dependencies:

    pip install "xyzspaces[geo]"
  • Install from its source repository on GitHub:

    pip install -e git+https://github.com/heremaps/xyz-spaces-python#egg=xyzspaces

If you want to run the test suite or experiment with the example notebooks bundled, you need to clone the whole repository:

  • Make a local clone of the repository hosting this package. The following command should do:

    git clone https://github.com/heremaps/xyz-spaces-python.git
  • Change into the repo root directory:

    cd xyzspaces

Interactive Map Layers

The xyzspaces package supports Interactive Map Layers which is Data Hub on HERE Platform. Using xyzspaces you can interact with your Interactive Map Layers using higher level pythonic interface that wraps the RESTful API. With Interactive Map Layers, data is stored in GeoJSON and can be retrieved dynamically at any zoom level. Interactive map layer is optimized for the visualization, analysis, and modification of data on a map (i.e., GIS functions).

Key features of Interactive Map Layers include:

  • Creating and modifying maps manually or programmatically; edits are published real-time and require no additional interaction.
  • Modifying data a granular feature and feature property level.
  • Adding and removing points, lines, and polygons directly on a map.
  • Ability to retrieve data in different tiling schemes.
  • Exploring and retrieving data by feature ID, bounding box, spatial search, property search, and features contained within a tile.
  • Searching for data by values of feature properties (e.g., speed limits, type of place, address, name, etc.).
  • Data sampling, making it possible to efficiently render an excerpt of a very large data set for visual reference and analysis.
  • Clustering using hexbins or quadbins to produce rich, visual data representations.

Credentials

To interact with Interactive Map Layer you will need an account on the HERE Platform. To get more details on the HERE Platform account please check our documentation Get a HERE account. Once you have the account follow the below steps to get credentials:

The HERE platform generated app credentials should look similar to the example below:

  here.user.id = <example_here>
  here.client.id = <example_here>
  here.access.key.id = <example_here>
  here.access.key.secret = <example_here>
  here.token.endpoint.url = <example_here>

You can provide your credentials using any of the following methods:

  • Default credentials
  • Environment variables
  • Credentials file

Default credentials

Place the credentials file into

For Linux/MacOS: $HOME/.here/credentials.properties

For Windows: %USERPROFILE%\.here\credentials.properties

Environment Variables

You can override default credentials by assigning values to the following environment variables:

HERE_USER_ID
HERE_CLIENT_ID
HERE_ACCESS_KEY_ID
HERE_ACCESS_KEY_SECRET
HERE_TOKEN_ENDPOINT_URL

Credentials File

You can specify any credentials file as an alternative to that found in ~/.here/credentials.properties. An error is generated if there is no file present at the path, or if the file is not properly formatted.

Documentation

Documentation is hosted here.

To build the docs locally run:

bash scripts/build_docs.sh

Hello World Example

The following are tiny "Hello World"-like examples that you can run to have a successful first XYZ experience right after installation!

Data Hub

import geojson
import os
import xyzspaces

os.environ["XYZ_TOKEN"] = "MY_XYZ_TOKEN"
xyz = xyzspaces.XYZ()

# Create a New Space
title = "My Demo Space"
description = "My Description"
space = xyz.spaces.new(title=title, description=description)

# Define a New Feature
feature =  {
    "type": "Feature",
    "properties": {"party": "Republican"},
    "geometry": {
        "type": "Polygon",
        "coordinates": [[
            [-104.05, 48.99],
            [-97.22,  48.98],
            [-96.58,  45.94],
            [-104.03, 45.94],
            [-104.05, 48.99]
        ]]
    }
}

# Save it to a Space and get its ID
feature_id = space.add_features(features=geojson.FeatureCollection([feature]))["features"][0]["id"]

# Read a Feature from a Space
feature = space.get_feature(feature_id=feature_id)
print(geojson.dumps(feature, indent=4, sort_keys=True))

Interactive Map Layer

import geojson
from xyzspaces import IML
from xyzspaces.iml.credentials import Credentials

credentials = Credentials.from_default() # credentials are in either credentials file at default location or in environment variables

layer_details = {
    "id": "demo-interactive-layer",
    "name": "Demo Interactive Layer",
    "summary": "Demo Interactive Layer",
    "description": "Demo Interactive Layer",
    "layerType": "interactivemap",
    "interactiveMapProperties": {},
}

iml = IML.new(
    catalog_id="demo-catalog1",
    catalog_name="demo-catalog",
    catalog_summary="Demo catalog",
    catalog_description="Demo catalog",
    layer_details=layer_details,
    credentials=credentials,
)

# Define a New Feature
feature = {
    "type": "Feature",
    "properties": {"party": "Republican"},
    "geometry": {
        "type": "Polygon",
        "coordinates": [
            [
                [-104.05, 48.99],
                [-97.22, 48.98],
                [-96.58, 45.94],
                [-104.03, 45.94],
                [-104.05, 48.99],
            ]
        ],
    },
}
# Save feature to interactive map layer
iml.layer.write_feature(feature_id="demo_feature", data=feature)

# Read feature from nteractive map layer
resp = iml.layer.get_feature(feature_id="demo_feature")
print(geojson.dumps(resp.to_geojson(), indent=4, sort_keys=True))

License

Copyright (C) 2019-2021 HERE Europe B.V.

Unless otherwise noted in LICENSE files for specific directories, the LICENSE in the root applies to all content in this repository.

Comments
  • Outdated package on conda-forge?

    Outdated package on conda-forge?

    Hi guys, I wanted to let you know that I recently installed xyzspace package from conda-forge (version '0.4.0') and it looks that some functionalities are missing there. E.g. according to the documentation one of the parameters of space.spatial_search() is 'force_2d' (to skip Z coordinate in the response). When I wanted to use it I got: TypeError: spatial_search() got an unexpected keyword argument 'force_2d'

    Thanks, Piotr

    no-issue-activity 
    opened by pioboch 4
  • Feature request: 'params' query for feature filtering

    Feature request: 'params' query for feature filtering

    Please implement the functionality to filter feature based in the Python client. A feature filter contains 3 components: property name, value and operator. The current implementation is not sufficient when one want to filter feature using an operator other than equal '='

    Feature filtering is realized in XYZ API via 'params' query for several endpoint, for example for tile request https://xyz.api.here.com/hub/static/swagger/#/Read%20Features/getFeaturesByTile

    Query Syntax ?p.property_name_1=value_1,value_2

    Supported operators

    "=" - equals
    "!=" - not equals
    ">=" or "=gte=" - greater than or equals
    "<=" or "=lte=" - less than or equals
    ">" or "=gt=" - greater than
    "<" or "=lt=" - less than
    "@>" or "=cs=" - contains
    

    Thank you

    enhancement 
    opened by minff 4
  • Exclude tests submodules from package

    Exclude tests submodules from package

    With current setup.py, pip install would still install "tests" package at the same level as xyzspaces package. Added wildcard pattern to exclude it

    opened by minff 3
  • Add initial ADRs

    Add initial ADRs

    Initial stab at adding ADRs, as described by Michael Nygard. This is in MD, but later down the line we might explore way to include this into the main documentation which is formatted in ReST...

    opened by deeplook 3
  • Added changes for custom url for self hosted data hub instances

    Added changes for custom url for self hosted data hub instances

    Signed-off-by: Kharude, Sachin [email protected]

    This PR includes changes for private instances of DataHub APIs. With this change, the user can provide a different base URL for self-hosted Data Hub APIs.

    opened by sackh 2
  • Consider suppressing returning resources for PUT/POST requests

    Consider suppressing returning resources for PUT/POST requests

    Using Accept: application/x-empty in PUT/POST feature requests would prevent the resource to be returned again in the HTTP response which would save some bandwidth and might even reduce server load a bit. If the user really wants the resource it can be obtained with a follow-up GET request.

    enhancement no-issue-activity 
    opened by deeplook 2
  • Resolution param for hexbin clusters has no effect

    Resolution param for hexbin clusters has no effect

    Sample snippet:

    from ipyleaflet import GeoJSON, Map
    from turfpy.measurement import center
    from xyzspaces import XYZ
    from xyzspaces.datasets import get_countries_data
    
    xyz_pro_token = "******"
    xyz = XYZ(credentials=xyz_pro_token)
    space = xyz.spaces.new(title="Hexbin Cluster Demo", description="Hexbin Cluster Demo")
    afg = get_countries_data()["features"][0]
    space.add_features(features=geojson.FeatureCollection([afg]))
    try:
        fc = None
        # Allowed params: [resolution, relativeResolution, absoluteResolution,
        #                  property, pointmode, countmode, noBuffer]
        fc = space.cluster("hexbin", clustering_params={"absoluteResolution": 5})
    finally:
        space.delete()
    
    c = list(reversed(center(afg)["geometry"]["coordinates"]))
    m = Map(center=c, zoom=5)
    if fc:
        m += GeoJSON(data=fc)
    m
    
    opened by deeplook 2
  • Added clientId in query params for Hub API requests

    Added clientId in query params for Hub API requests

    Signed-off-by: Kharude, Sachin [email protected]

    • Added clientId in query params as suggested by DataHub developers this helps in identifying requests sent from xyzspaces library.
    • Minor variable name changes as per pep8 guidelines.
    • Improved clean up for activity log test.
    opened by sackh 2
  • Fix: added mode and viz_sampling params in space class method features_in_tile

    Fix: added mode and viz_sampling params in space class method features_in_tile

    This PR fixes two missing parameters in the Space class's method features_in_tile, mode and viz_sampling which were present in the Api class's method get_space_tile. Also added missing proclamation changes.

    opened by sackh 1
Releases(v0.7.2)
  • v0.7.2(Aug 18, 2021)

  • v0.7.1(Aug 10, 2021)

  • v0.7.0(Aug 10, 2021)

  • v0.6.0(Jun 17, 2021)

  • 0.5.0(Feb 1, 2021)

    Features

    • Added functionality to clone Space. (#93)
    • Added support for the new force2D parameter for all the APIs used to read features. (#96)
    • Added support for new mode and vizSampling params in HubApi.get_space_tile. (#101)
    Source code(tar.gz)
    Source code(zip)
  • 0.4.0(Sep 18, 2020)

    Features

    • Upload data from kml and geobuff files to space.
    • Upload Geopandas Dataframe to space and read space data as Geopandas Dataframe.
    • Enabled property search while searching features in the bounding box.
    • Enabled property search while searching features in tile.
    • Improved performance of CSV and GeoJSON files upload.
    • Enabled conversion of original projection of shapefile to EPSG:4326.
    • New notebook illustrating spatial search on Microsoft US building footprints dataset.

    Fixes:

    • Fixed encoding and projections issue for shapefile upload.
    • Fixed duplicate features for spatial_search_geometry with the division.
    • Fixed upload of duplicate features while uploading to space using add_features.
    • Madedescription param optional when creating the space.
    • Added limit param to method iter_feature of Space class to control the number of features to iterate in a single call.
    Source code(tar.gz)
    Source code(zip)
  • 0.3.2(Aug 19, 2020)

    Features

    Upload Enhancements:

    • Supporting upload via shapefile to space. (#40)
    • Supporting upload via WKT file to space. (#41)
    • Supporting upload via gpx file to space. (#42)

    Optimized the spatial search to search features from large geometries. (#44)

    Misc

    • Added Binder support to the repository. (#28)
    Source code(tar.gz)
    Source code(zip)
  • 0.3.1(Jul 24, 2020)

  • 0.3(Jul 24, 2020)

Owner
HERE Technologies
HERE Technologies, the leading location cloud.
HERE Technologies
A Python framework for building geospatial web-applications

Hey there, this is Greppo... A Python framework for building geospatial web-applications. Greppo is an open-source Python framework that makes it easy

Greppo 304 Dec 27, 2022
Platform for building statistical models of cities and regions

UrbanSim UrbanSim is a platform for building statistical models of cities and regions. These models help forecast long-range patterns in real estate d

Urban Data Science Toolkit 419 Dec 30, 2022
Solving the Traveling Salesman Problem using Self-Organizing Maps

Solving the Traveling Salesman Problem using Self-Organizing Maps This repository contains an implementation of a Self Organizing Map that can be used

Diego Vicente 3.1k Dec 31, 2022
Logging the position of the car on an sdcard

audi-mmi-3g-gps-logging Logging the position of the car on an sdcard, startup script origin not clear to me, logging setup and time change is what I d

2 May 31, 2022
Tile Map Service and OGC Tiles API for QGIS Server

Tiles API Add tiles API to QGIS Server Tiles Map Service API OGC Tiles API Tile Map Service API - TMS The TMS API provides these URLs: /tms/? to get i

3Liz 6 Dec 01, 2021
A compilation of several single-beam bathymetry surveys of the Caribbean

Caribbean - Single-beam bathymetry This dataset is a compilation of several single-beam bathymetry surveys of the Caribbean ocean displaying a wide ra

Fatiando a Terra Datasets 0 Jan 20, 2022
Geodata extensions for Django REST Framework

Django-Spillway Django and Django REST Framework integration of raster and feature based geodata. Spillway builds on the immensely marvelous Django RE

Brian Galey 62 Jan 04, 2023
Helping data scientists better understand their datasets and models in text classification. With love from ServiceNow.

Azimuth, an open-source dataset and error analysis tool for text classification, with love from ServiceNow. Overview Azimuth is an open source applica

ServiceNow 145 Dec 23, 2022
A bot that tweets info and location map for new bicycle parking added to OpenStreetMap within a GeoJSON boundary.

Bike parking tweepy bot app A twitter bot app that searches for bicycle parking added to OpenStreetMap. Relies on AWS Lambda/S3, Python3, Tweepy, Flas

Angelo Trivisonno 1 Dec 19, 2021
Django model field that can hold a geoposition, and corresponding widget

django-geoposition A model field that can hold a geoposition (latitude/longitude), and corresponding admin/form widget. Prerequisites Starting with ve

Philipp Bosch 324 Oct 17, 2022
Pure python WMS

Ogcserver Python WMS implementation using Mapnik. Depends Mapnik = 0.7.0 (and python bindings) Pillow PasteScript WebOb You will need to install Map

Mapnik 130 Dec 28, 2022
Implementation of Trajectory classes and functions built on top of GeoPandas

MovingPandas MovingPandas implements a Trajectory class and corresponding methods based on GeoPandas. Visit movingpandas.org for details! You can run

Anita Graser 897 Jan 01, 2023
Python bindings to libpostal for fast international address parsing/normalization

pypostal These are the official Python bindings to https://github.com/openvenues/libpostal, a fast statistical parser/normalizer for street addresses

openvenues 651 Dec 16, 2022
Zora is a python program that searches for GeoLocation info for given CIDR networks , with options to search with API or without API

Zora Zora is a python program that searches for GeoLocation info for given CIDR networks , with options to search with API or without API Installing a

z3r0day 1 Oct 26, 2021
Map Ookla server locations as a Kernel Density Estimation (KDE) geographic map plot.

Ookla Server KDE Plotting This notebook was created to map Ookla server locations as a Kernel Density Estimation (KDE) geographic map plot. Currently,

Jonathan Lo 1 Feb 12, 2022
Centroids as a Service

Centroids! This application reads a valid geojson FeatureCollection and returns a valid geojson FeatureColleciton of centroids. In the output: All pro

Lyzi Diamond 20 Aug 29, 2021
A Python tool to display geolocation information in the traceroute.

IP2Trace Python IP2Trace Python is a Python tool allowing user to get IP address information such as country, region, city, latitude, longitude, zip c

IP2Location 22 Jan 08, 2023
Tools for the extraction of OpenStreetMap street network data

OSMnet Tools for the extraction of OpenStreetMap (OSM) street network data. Intended to be used in tandem with Pandana and UrbanAccess libraries to ex

Urban Data Science Toolkit 47 Sep 21, 2022
Imperial Valley Geomorphology Map

Roughly maps the extent of basins, basin edges, and mountains in the Imperial Valley by grouping terrain classes from the Iwahashi et al. 2021 California terrian classification model.

0 Dec 13, 2022
Python project to generate Kerala's distrcit level panchayath map.

Kerala-Panchayath-Maps Python project to generate Kerala's distrcit level panchayath map. As of now, geojson files of Kollam and Kozhikode are added t

Athul R T 2 Jan 10, 2022