Vector tile server for the Wildfire Predictive Services Unit

Overview

wps-tileserver

Vector tile server for the Wildfire Predictive Services Unit

Lifecycle:Experimental

Overview

The intention of this project is to:

  • provide tools to easily spin up a vector tile server in openshift, in a project agnostic manner.
  • provide tools to manually pull data from an esri arc server into a postgis database.
  • provide tools that periodically synchronize data from an esri arc server into postgis.

Components

  • postgis database server. (it is assumed you have a working postgis database server)
  • pg_tileserv - serves up vector tiles from postgis server.
  • proxy server (varnish?) - caches responses.
  • sync cronjob - updates database periodically.

Reference

https://blog.crunchydata.com/blog/production-postgis-vector-tiles-caching https://github.com/CrunchyData/pg_tileserv

Local development

Assumptions

  • postgresql server with postgis running locally

Configure pg_tile server

Download the latest pg_tileserver, unzip and start.

mkdir pg_tileserv
cd pg_tileserv
wget https://postgisftw.s3.amazonaws.com/pg_tileserv_latest_linux.zip
unzip pg_tileserv
export DATABASE_URL=postgresql://tileserv:[email protected]/tileserv
./pg_tileserv

Install binary requirements

Install gdal

Ubuntu

sudo apt install gdal-bin

Install python requirements

This step only required if you're going to be using the python scripts in this repo to load data. If you're loading directly from shapefiles, then skip this step.

Assumptions

  • appropriate python version is install
  • python poetry is installed

Install python requirements

poetry install

Loading data

Create a user and database for your tileserver

create user tileserv with password 'tileserv';
create database tileserv with owner tileserv;
\c tileserv
CREATE EXTENSION IF NOT EXISTS postgis;

Using an arcserver rest endpoint

Given some arcserver layer endpoint, e.g.: https://maps.gov.bc.ca/arcserver/rest/services/whse/bcgw_pub_whse_legal_admin_boundaries/MapServer/8

poetry run python fetch_feature_layer.py https://maps.gov.bc.ca/arcserver/rest/services/whse/bcgw_pub_whse_legal_admin_boundaries/MapServer/8

Using a shapefile

ogr2ogr -f "PostgreSQL" PG:"dbname=tileserv host=localhost user=tileserv password=tileserv" "my_shapefile.shp" -lco precision=NO -nln fire_area_thessian_polygons

Deploy

Assumptions

  • You have the oc command line installed and you're logged in.
  • You have docker installed locally.
  • You have a postgres database in your target openshift environment that can be accessed by pg_tileserv (you made need to add additional rules to allow your tile server to communicate with your database.)

Instructions

Prepare your openshift environment

# we have docker limits, so pull the images local - then put them in openshift

# pull local
docker pull eeacms/varnish
docker pull pramsey/pg_tileserv

# tag for upload
docker tag eeacms/varnish image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/varnish:latest
docker tag pramsey/pg_tileserv image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/pg_tileserv:latest

# log in to openshift docker
docker login -u developer -p $(oc whoami -t) image-registry.apps.silver.devops.gov.bc.ca

# push it
docker push image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/varnish:latest
docker push image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/pg_tileserv:latest

Deploy pg_tilserver

# deploy pg_tileserv
oc -n e1e498-dev process -f tileserv.yaml | oc -n e1e498-dev apply -f -

Manually Loading data into your openshift hosted postgis database

The easiest way to achieve this, is to tunnel to your database server and then run the import scripts as if your database was local.

oc port-forward patroni-wps-mapserver-prototype-1 5432:5432
Owner
Province of British Columbia
Where Ideas Work
Province of British Columbia
You can change your mac address with this program.

1 - Warning! You can use this program with Kali Linux. Therefore if you don't install the Kali Linux. Firstly you need to install Kali Linux. 2 - Star

Mustafa Bahadır Doğrusöz 1 Jun 10, 2022
Jack Morgan's Advent of Code Solutions

Advent-of-Code Jack Morgan's Advent of Code Solutions Usage Run . initiate.sh year day To initiate a day. This sets up a template python file, and pul

Jack Morgan 1 Dec 10, 2021
A Classroom Engagement Platform

Project Introduction This is project introduction Setup Setting up Postgres This is the most tricky part when setting up the application. You will nee

Santosh Kumar Patro 1 Nov 18, 2021
Discover and load entry points from installed packages

Entry points are a way for Python packages to advertise objects with some common interface. The most common examples are console_scripts entry points,

Thomas Kluyver 69 Jul 05, 2022
A light library to build tiny websites

A light library to build tiny websites

BT.Q 1 Dec 23, 2021
CALPHAD tools for designing thermodynamic models, calculating phase diagrams and investigating phase equilibria.

CALPHAD tools for designing thermodynamic models, calculating phase diagrams and investigating phase equilibria.

pycalphad 189 Dec 13, 2022
A proof-of-concept package manager for Cairo contracts/libraries

glyph A proof-of-concept package manager for Cairo contracts/libraries. Distribution through pypi. Installation through existing package managers -- p

Sam Barnes 11 Jun 06, 2022
Convert Roman numerals to modern numerals and vice-versa

Roman Numeral Conversion Utilities This is a utility module for converting from and to Roman numerals. It supports numbers upto 3,999,999, using the v

Fictive Kin 1 Dec 17, 2021
A function decorator for enforcing function signatures

A function decorator for enforcing function signatures

Emmanuel I. Obi 0 Dec 08, 2021
SMS-b0mber VANDALIZM developed for VK group

VANDALIZM SMS-b0mber VANDALIZM developed for VK group https://vk.com/dark__code if you come across this code, you can use it for your own purposes) ус

5 Jun 24, 2022
A StarkNet project template based on a Pythonic environment

StarkNet Project Template This is an opinionated StarkNet project template. It is based around the Python's ecosystem and best practices. tox to manag

Francesco Ceccon 5 Apr 21, 2022
Remote execution of a simple function on the server

FunFetch Remote execution of a simple function on the server All types of Python support objects.

Decave 4 Jun 30, 2022
A program to calculate the are of a triangle. made with Python.

Area-Calculator What is Area-Calculator? Area-Calculator is a program to find out the area of a triangle easily. fully made with Python. Needed a pyth

Chandula Janith 0 Nov 27, 2021
Data Orchestration Platform

Table of contents What is DOP Design Concept A Typical DOP Orchestration Flow Prerequisites - Run in Docker For DOP Native Features For DBT Instructio

Datatonic 61 Mar 04, 2022
The-White-Noise-Project - The project creates noise intentionally

The-White-Noise-Project High quality audio matters everywhere, even in noise. Be

Ali Hakim Taşkıran 1 Jan 02, 2022
This application is made solely for entertainment purposes

Timepass This application is made solely for entertainment purposes helps you find things to do when you're bored ! tells jokes guaranteed to bring on

Omkar Pramod Hankare 2 Nov 24, 2021
InverterApi - This project has been designed to take monitoring data from Voltronic, Axpert, Mppsolar PIP, Voltacon, Effekta

InverterApi - This project has been designed to take monitoring data from Voltronic, Axpert, Mppsolar PIP, Voltacon, Effekta

Josep Escobar 2 Sep 03, 2022
Change ACLs for QNAP LXD unprivileged container.

qnaplxdunpriv If Advanced Folder Permissions is enabled in QNAP NAS, unprivileged LXD containers won't start. qnaplxdunpriv changes ACLs of some Conta

1 Jan 10, 2022
MeepoBenchmark - This project aims at providing the scripts, logs, and analytic results for Meepo Blockchain

MeepoBenchmark - This project aims at providing the scripts, logs, and analytic results for Meepo Blockchain

Peilin Zheng 3 Aug 16, 2022
Here, I have discuss the three methods of list reversion. The three methods are built-in method, slicing method and position changing method.

Three-different-method-for-list-reversion Here, I have discuss the three methods of list reversion. The three methods are built-in method, slicing met

Sachin Vinayak Dabhade 4 Sep 24, 2021