Imports VZD (Latvian State Land Service) open data into postgis enabled database

Overview

Python script main.py downloads and imports Latvian addresses into PostgreSQL database. Data contains parishes, counties, cities, towns, and streets.

Shell script import-kadastrs.sh downloads and imports parcels shapefiles (cadastral groups, buildings, engineering structures, parcels, parcel borders, parcel errors, parcel parts, surveying statuses, and way restrictions)..

Requirements

  • PostgreSQL with PostGIS extension
  • Python3 with httpx and psycopg2 modules
  • shp2pgsql (shipped with PostGIS), curl, and jq command-line JSON processor for parcel import script.

Possible future work

  • Add import of addresses related shapefiles (same dataset, different archive file; contributors welcome, I have no use case for it just now)
  • Add a utility script to download and import of parcels data (there's a lot of shapefiles)

Contributions

These are welcome (use issues to report or discuss, pull requests to implement).

Usage

To fetch and import addresses, create schema (psql schema ) and then run VZD_DBNAME=schema python3 main.py --verbose.

To download and import all parcel shapefiles, run VZD_DBNAME=schema ./import-kadastrs.sh. This will take a while.

Behaviour

Addresses import

Python script checks against Latvian address register open data , using If-Modified-Since header, which it saves to a file for future reference, so data is being downloaded only if it has been updated. This means that it can be dropped into cron job to download data when it is updated.

If data has been downloaded, it's unzipped into data/csv and then imported into PostgreSQL. Schema has to be created (it can be found in schema.sql)

If data has invalid coordinates (latitude or longitude is not a number), it's skipped.

For aw_eka table column geom is created, and an spatial index is added. SRID 4326 is used, so some offsets may arise.

Parcels import

Shell script does nothing fancy. It just fetches JSON metadata, extracts list of all shapefile zip's, downloads them, then imports into database (via dropping, creating and populating tables). Spatial index on geom is also created.

Data

Data has been released by the State Land Service of the Republic of Latvia under goverment's OpenData initiative, and is available at this data.gov.lv page . It's released under CC BY 4.0 license.

Spec for address data , spec for parcel data.

License

CC BY 4.0, which means that it can be used for free, however attribution is required, and no additional restrictions on this data can be imposed. This script follows suit.

Owner
Kaspars Foigts
Kaspars Foigts
Geocoding library for Python.

geopy geopy is a Python client for several popular geocoding web services. geopy makes it easy for Python developers to locate the coordinates of addr

geopy 3.8k Dec 30, 2022
Python interface to PROJ (cartographic projections and coordinate transformations library)

pyproj Python interface to PROJ (cartographic projections and coordinate transformations library). Documentation Stable: http://pyproj4.github.io/pypr

832 Dec 31, 2022
LicenseLocation - License Location With Python

LicenseLocation Hi,everyone! ❤ 🧡 💛 💚 💙 💜 This is my first project! ✔ Actual

The Bin 1 Jan 25, 2022
WebGL2 powered geospatial visualization layers

deck.gl | Website WebGL2-powered, highly performant large-scale data visualization deck.gl is designed to simplify high-performance, WebGL-based visua

Vis.gl 10.5k Jan 08, 2023
How to use COG's (Cloud optimized GeoTIFFs) with Rasterio

How to use COG's (Cloud optimized GeoTIFFs) with Rasterio According to Cogeo.org: A Cloud Opdtimized GeoTIFF (COG) is a regular GeoTIFF file, aimed at

Marvin Gabler 8 Jul 29, 2022
ArcGIS Python Toolbox for WhiteboxTools

WhiteboxTools-ArcGIS ArcGIS Python Toolbox for WhiteboxTools. This repository is related to the ArcGIS Python Toolbox for WhiteboxTools, which is an A

Qiusheng Wu 190 Dec 30, 2022
Tool to suck data from ArcGIS Server and spit it into PostgreSQL

chupaESRI About ChupaESRI is a Python module/command line tool to extract features from ArcGIS Server map services. Name? Think "chupacabra" or "Chupa

John Reiser 34 Dec 04, 2022
Spatial Interpolation Toolbox is a Python-based GUI that is able to interpolate spatial data in vector format.

Spatial Interpolation Toolbox This is the home to Spatial Interpolation Toolbox, a graphical user interface (GUI) for interpolating geographic vector

Michael Ward 2 Nov 01, 2021
Tile Map Service and OGC Tiles API for QGIS Server

Tiles API Add tiles API to QGIS Server Tiles Map Service API OGC Tiles API Tile Map Service API - TMS The TMS API provides these URLs: /tms/? to get i

3Liz 6 Dec 01, 2021
Interactive Maps with Geopandas

Create Interactive maps 🗺️ with your geodataframe Geopatra extends geopandas for interactive mapping and attempts to wrap the goodness of amazing map

sangarshanan 46 Aug 16, 2022
A compilation of several single-beam bathymetry surveys of the Caribbean

Caribbean - Single-beam bathymetry This dataset is a compilation of several single-beam bathymetry surveys of the Caribbean ocean displaying a wide ra

Fatiando a Terra Datasets 0 Jan 20, 2022
scalable analysis of images and time series

thunder scalable analysis of image and time series analysis in python Thunder is an ecosystem of tools for the analysis of image and time series data

thunder-project 813 Dec 29, 2022
A toolbox for processing earth observation data with Python.

eo-box eobox is a Python package with a small collection of tools for working with Remote Sensing / Earth Observation data. Package Overview So far, t

13 Jan 06, 2022
Open GeoJSON data on geojson.io

geojsonio.py Open GeoJSON data on geojson.io from Python. geojsonio.py also contains a command line utility that is a Python port of geojsonio-cli. Us

Jacob Wasserman 114 Dec 21, 2022
Replace MSFS2020's bing map to google map

English verison here 中文 免责声明 本教程提到的方法仅用于研究和学习用途。我不对使用、拓展该教程及方法所造成的任何法律责任和损失负责。 背景 微软模拟飞行2020的地景使用了Bing的卫星地图,然而卫星地图比较老旧,很多地区都是几年前的图设置直接是没有的。这种现象在全球不同地区

hesicong 272 Dec 24, 2022
EOReader is a multi-satellite reader allowing you to open optical and SAR data.

Remote-sensing opensource python library reading optical and SAR sensors, loading and stacking bands, clouds, DEM and index.

ICube-SERTIT 152 Dec 30, 2022
A light-weight, versatile XYZ tile server, built with Flask and Rasterio :earth_africa:

Terracotta is a pure Python tile server that runs as a WSGI app on a dedicated webserver or as a serverless app on AWS Lambda. It is built on a modern

DHI GRAS 531 Dec 28, 2022
Python project to generate Kerala's distrcit level panchayath map.

Kerala-Panchayath-Maps Python project to generate Kerala's distrcit level panchayath map. As of now, geojson files of Kollam and Kozhikode are added t

Athul R T 2 Jan 10, 2022
A service to auto provision devices in Aruba Central based on the Geo-IP location

Location Based Provisioning Service for Aruba Central A service to auto provision devices in Aruba Central based on the Geo-IP location Geo-IP auto pr

Will Smith 3 Mar 22, 2022
A GUI widget for Linux to show current time in different timezones.

A GUI widget to show current time in different timezones (under development). To use this widget: Run scripts/startup.py Select a country. A list of t

B.Jothin kumar 11 Nov 10, 2022