Imports VZD (Latvian State Land Service) open data into postgis enabled database

Overview

Python script main.py downloads and imports Latvian addresses into PostgreSQL database. Data contains parishes, counties, cities, towns, and streets.

Shell script import-kadastrs.sh downloads and imports parcels shapefiles (cadastral groups, buildings, engineering structures, parcels, parcel borders, parcel errors, parcel parts, surveying statuses, and way restrictions)..

Requirements

  • PostgreSQL with PostGIS extension
  • Python3 with httpx and psycopg2 modules
  • shp2pgsql (shipped with PostGIS), curl, and jq command-line JSON processor for parcel import script.

Possible future work

  • Add import of addresses related shapefiles (same dataset, different archive file; contributors welcome, I have no use case for it just now)
  • Add a utility script to download and import of parcels data (there's a lot of shapefiles)

Contributions

These are welcome (use issues to report or discuss, pull requests to implement).

Usage

To fetch and import addresses, create schema (psql schema ) and then run VZD_DBNAME=schema python3 main.py --verbose.

To download and import all parcel shapefiles, run VZD_DBNAME=schema ./import-kadastrs.sh. This will take a while.

Behaviour

Addresses import

Python script checks against Latvian address register open data , using If-Modified-Since header, which it saves to a file for future reference, so data is being downloaded only if it has been updated. This means that it can be dropped into cron job to download data when it is updated.

If data has been downloaded, it's unzipped into data/csv and then imported into PostgreSQL. Schema has to be created (it can be found in schema.sql)

If data has invalid coordinates (latitude or longitude is not a number), it's skipped.

For aw_eka table column geom is created, and an spatial index is added. SRID 4326 is used, so some offsets may arise.

Parcels import

Shell script does nothing fancy. It just fetches JSON metadata, extracts list of all shapefile zip's, downloads them, then imports into database (via dropping, creating and populating tables). Spatial index on geom is also created.

Data

Data has been released by the State Land Service of the Republic of Latvia under goverment's OpenData initiative, and is available at this data.gov.lv page . It's released under CC BY 4.0 license.

Spec for address data , spec for parcel data.

License

CC BY 4.0, which means that it can be used for free, however attribution is required, and no additional restrictions on this data can be imposed. This script follows suit.

Owner
Kaspars Foigts
Kaspars Foigts
iNaturalist observations along hiking trails

This tool reads the route of a hike and generates a table of iNaturalist observations along the trails. It also shows the observations and the route of the hike on a map. Moreover, it saves waypoints

7 Nov 11, 2022
🌐 Local tile server for viewing geospatial raster files with ipyleaflet

🌐 Local Tile Server for Geospatial Rasters Need to visualize a rather large raster (gigabytes) you have locally? This is for you. A Flask application

Bane Sullivan 192 Jan 04, 2023
Centroids as a Service

Centroids! This application reads a valid geojson FeatureCollection and returns a valid geojson FeatureColleciton of centroids. In the output: All pro

Lyzi Diamond 20 Aug 29, 2021
A Python package for delineating nested surface depressions from digital elevation data.

Welcome to the lidar package lidar is Python package for delineating the nested hierarchy of surface depressions in digital elevation models (DEMs). I

Qiusheng Wu 166 Jan 03, 2023
This program analizes films database with adresses, and creates a folium map with closest films to the coordinates

Films-map-project UCU CS lab 1.2, 1st year This program analizes films database with adresses, and creates a folium map with closest films to the coor

Artem Moskovets 1 Feb 09, 2022
Color correction plugin for rasterio

rio-color A rasterio plugin for applying basic color-oriented image operations to geospatial rasters. Goals No heavy dependencies: rio-color is purpos

Mapbox 111 Nov 15, 2022
A light-weight, versatile XYZ tile server, built with Flask and Rasterio :earth_africa:

Terracotta is a pure Python tile server that runs as a WSGI app on a dedicated webserver or as a serverless app on AWS Lambda. It is built on a modern

DHI GRAS 531 Dec 28, 2022
LEOGPS - Satellite Navigation with GPS on Python!

LEOGPS is an open-source Python software which performs relative satellite navigation between two formation flying satellites, with the objective of high accuracy relative positioning. Specifically,

Samuel Low 50 Dec 13, 2022
A simple reverse geocoder that resolves a location to a country

Reverse Geocoder This repository holds a small web service that performs reverse geocoding to determine whether a user specified location is within th

4 Dec 25, 2021
Build, deploy and extract satellite public constellations with one command line.

SatExtractor Build, deploy and extract satellite public constellations with one command line. Table of Contents About The Project Getting Started Stru

Frontier Development Lab 70 Nov 18, 2022
Streamlit Component for rendering Folium maps

streamlit-folium This Streamlit Component is a work-in-progress to determine what functionality is desirable for a Folium and Streamlit integration. C

Randy Zwitch 224 Dec 30, 2022
A Python interface between Earth Engine and xarray

eexarray A Python interface between Earth Engine and xarray Description eexarray was built to make processing gridded, mesoscale time series data quic

Aaron Zuspan 159 Dec 23, 2022
Open Data Cube analyses continental scale Earth Observation data through time

Open Data Cube Core Overview The Open Data Cube Core provides an integrated gridded data analysis environment for decades of analysis ready earth obse

Open Data Cube 410 Dec 13, 2022
A proof-of-concept jupyter extension which converts english queries into relevant python code

Text2Code for Jupyter notebook A proof-of-concept jupyter extension which converts english queries into relevant python code. Blog post with more deta

DeepKlarity 2.1k Dec 29, 2022
Xarray backend to Copernicus Sentinel-1 satellite data products

xarray-sentinel WARNING: this product is a "technology preview" / pre-Alpha Xarray backend to explore and load Copernicus Sentinel-1 satellite data pr

B-Open 191 Dec 15, 2022
framework for large-scale SAR satellite data processing

pyroSAR A Python Framework for Large-Scale SAR Satellite Data Processing The pyroSAR package aims at providing a complete solution for the scalable or

John Truckenbrodt 389 Dec 21, 2022
Google Maps keeps old satellite imagery around for a while – this tool collects what's available for a user-specified region in the form of a GIF.

google-maps-at-88-mph The folks maintaining Google Maps regularly update the satellite imagery it serves its users, but outdated versions of the image

Noah Doersing 111 Sep 27, 2022
Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

django-rest-framework-gis Geographic add-ons for Django Rest Framework - Mailing List. Install last stable version from pypi pip install djangorestfra

OpenWISP 981 Jan 03, 2023
Pandas Network Analysis: fast accessibility metrics and shortest paths, using contraction hierarchies :world_map:

Pandana Pandana is a Python library for network analysis that uses contraction hierarchies to calculate super-fast travel accessibility metrics and sh

Urban Data Science Toolkit 321 Jan 05, 2023
Python interface to PROJ (cartographic projections and coordinate transformations library)

pyproj Python interface to PROJ (cartographic projections and coordinate transformations library). Documentation Stable: http://pyproj4.github.io/pypr

832 Dec 31, 2022