Vector tile server for the Wildfire Predictive Services Unit

The intention of this project is to:

• provide tools to easily spin up a vector tile server in openshift, in a project agnostic manner.
• provide tools to manually pull data from an esri arc server into a postgis database.
• provide tools that periodically synchronize data from an esri arc server into postgis.

• postgis database server. (it is assumed you have a working postgis database server)
• pg_tileserv - serves up vector tiles from postgis server.
• proxy server (varnish?) - caches responses.
• sync cronjob - updates database periodically.

https://blog.crunchydata.com/blog/production-postgis-vector-tiles-caching https://github.com/CrunchyData/pg_tileserv

• postgresql server with postgis running locally

Download the latest pg_tileserver, unzip and start.

mkdir pg_tileserv
cd pg_tileserv
wget https://postgisftw.s3.amazonaws.com/pg_tileserv_latest_linux.zip
unzip pg_tileserv
export DATABASE_URL=postgresql://tileserv:tileserv@localhost/tileserv
./pg_tileserv

This step only required if you're going to be using the python scripts in this repo to load data. If you're loading directly from shapefiles, then skip this step.

• appropriate python version is installed
• python poetry is installed

poetry install

create user tileserv with password 'tileserv';
create database tileserv with owner tileserv;
\c tileserv
CREATE EXTENSION IF NOT EXISTS postgis;

Given some arcserver layer endpoint, e.g.: Fire Zones or Fire Centres

Use the fetch_feature_layer.py helper script:

poetry run python fetch_feature_layer.py --help

e.g.:

poetry run python fetch_feature_layer.py https://maps.gov.bc.ca/arcserver/rest/services/whse/bcgw_pub_whse_legal_admin_boundaries/MapServer/8 localhost tileserv tileserv tileserv fire_zones

-nlt Define the geometry type for the created layer -lco NAME=VALUE Layer creation option (format specific) -nln Assign an alternate name to the new layer -overwrite Delete the output layer and recreate it empty

ogr2ogr -f "PostgreSQL" PG:"dbname=tileserv host=localhost user=tileserv password=tileserv" "my_shapefile.shp" -nlt MULTIPOLYGON -lco precision=NO -nln fire_area_thessian_polygons -overwrite

note - special characters in the ogr2ogr password string can be escaped with a backslash.

• You have the oc command line installed and you're logged in.
• You have docker installed locally.
• You have a postgres database in your target openshift environment that can be accessed by pg_tileserv (you made need to add additional rules to allow your tile server to communicate with your database.)

# we have docker limits, so pull the pg_tileserv images locally - then put them in openshift

# pull local
docker pull pramsey/pg_tileserv

# tag for upload
docker tag pramsey/pg_tileserv image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/pg_tileserv:latest

# log in to openshift docker
docker login -u developer -p $(oc whoami -t) image-registry.apps.silver.devops.gov.bc.ca

# push it
docker push image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/pg_tileserv:latest

# prepare nginx - creating a build configuration
# note: for some reason specifying the tag for nginx will result in an image that doesn't support s2i
oc new-build nginx~[git hub repository] --context-dir=[folder with nginx config] --name=[name of buildconfig and imagestream]
# e.g.: oc -n e1e498-tools new-build nginx~https://github.com/bcgov/wps-vector-tileserver.git --context-dir=openshift --name=nginx-tilecache

# deploy pg_tileserv
oc -n e1e498-dev process -f tileserv.yaml | oc -n e1e498-dev apply -f -

The easiest way to achieve this, is to tunnel to your database server and then run the import scripts as if your database was local.

oc port-forward patroni-wps-mapserver-prototype-1 5432:5432