Vector tile server for the Wildfire Predictive Services Unit

Overview

wps-tileserver

Vector tile server for the Wildfire Predictive Services Unit

Lifecycle:Experimental

Overview

The intention of this project is to:

  • provide tools to easily spin up a vector tile server in openshift, in a project agnostic manner.
  • provide tools to manually pull data from an esri arc server into a postgis database.
  • provide tools that periodically synchronize data from an esri arc server into postgis.

Components

  • postgis database server. (it is assumed you have a working postgis database server)
  • pg_tileserv - serves up vector tiles from postgis server.
  • proxy server (varnish?) - caches responses.
  • sync cronjob - updates database periodically.

Reference

https://blog.crunchydata.com/blog/production-postgis-vector-tiles-caching https://github.com/CrunchyData/pg_tileserv

Local development

Assumptions

  • postgresql server with postgis running locally

Configure pg_tile server

Download the latest pg_tileserver, unzip and start.

mkdir pg_tileserv
cd pg_tileserv
wget https://postgisftw.s3.amazonaws.com/pg_tileserv_latest_linux.zip
unzip pg_tileserv
export DATABASE_URL=postgresql://tileserv:[email protected]/tileserv
./pg_tileserv

Install binary requirements

Install gdal

Ubuntu

sudo apt install gdal-bin

Install python requirements

This step only required if you're going to be using the python scripts in this repo to load data. If you're loading directly from shapefiles, then skip this step.

Assumptions

  • appropriate python version is install
  • python poetry is installed

Install python requirements

poetry install

Loading data

Create a user and database for your tileserver

create user tileserv with password 'tileserv';
create database tileserv with owner tileserv;
\c tileserv
CREATE EXTENSION IF NOT EXISTS postgis;

Using an arcserver rest endpoint

Given some arcserver layer endpoint, e.g.: https://maps.gov.bc.ca/arcserver/rest/services/whse/bcgw_pub_whse_legal_admin_boundaries/MapServer/8

poetry run python fetch_feature_layer.py https://maps.gov.bc.ca/arcserver/rest/services/whse/bcgw_pub_whse_legal_admin_boundaries/MapServer/8

Using a shapefile

ogr2ogr -f "PostgreSQL" PG:"dbname=tileserv host=localhost user=tileserv password=tileserv" "my_shapefile.shp" -lco precision=NO -nln fire_area_thessian_polygons

Deploy

Assumptions

  • You have the oc command line installed and you're logged in.
  • You have docker installed locally.
  • You have a postgres database in your target openshift environment that can be accessed by pg_tileserv (you made need to add additional rules to allow your tile server to communicate with your database.)

Instructions

Prepare your openshift environment

# we have docker limits, so pull the images local - then put them in openshift

# pull local
docker pull eeacms/varnish
docker pull pramsey/pg_tileserv

# tag for upload
docker tag eeacms/varnish image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/varnish:latest
docker tag pramsey/pg_tileserv image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/pg_tileserv:latest

# log in to openshift docker
docker login -u developer -p $(oc whoami -t) image-registry.apps.silver.devops.gov.bc.ca

# push it
docker push image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/varnish:latest
docker push image-registry.apps.silver.devops.gov.bc.ca/e1e498-tools/pg_tileserv:latest

Deploy pg_tilserver

# deploy pg_tileserv
oc -n e1e498-dev process -f tileserv.yaml | oc -n e1e498-dev apply -f -

Manually Loading data into your openshift hosted postgis database

The easiest way to achieve this, is to tunnel to your database server and then run the import scripts as if your database was local.

oc port-forward patroni-wps-mapserver-prototype-1 5432:5432
Owner
Province of British Columbia
Where Ideas Work
Province of British Columbia
Q-Tracker is originally a High School Project created by Admins of Cirus Lab.

Q-Tracker is originally a High School Project created by Admins of Cirus Lab. It's completly coded in python along with mysql.(Tkinter For GUI)

Adithya Krishnan 2 Nov 14, 2022
Users can read others' travel journeys in addition to being able to upload and delete posts detailing their own experiences

Users can read others' travel journeys in addition to being able to upload and delete posts detailing their own experiences! Posts are organized by country and destination within that country.

Christopher Zeas 1 Feb 03, 2022
easy_sbatch - Batch submitting Slurm jobs with script templates

easy_sbatch - Batch submitting Slurm jobs with script templates

Wei Shen 13 Oct 11, 2022
Would upload anything I do with/related to brainfuck

My Brainfu*k Repo Basically wanted to create something with Brainfu*k but realized that with the smol brain I have, I need to see the cell values real

Rafeed 1 Mar 22, 2022
Small pip update helpers.

pipdate pipdate is a collection of small pip update helpers. The command pipdate # or python3.9 -m pipdate updates all your pip-installed packages. (O

Nico Schlömer 69 Dec 18, 2022
Test to grab m3u from YouTube live.

YouTube_to_m3u https://raw.githubusercontent.com/benmoose39/YouTube_to_m3u/main/youtube.m3u Updated m3u links of YouTube live channels, auto-updated e

136 Jan 06, 2023
This is the Quiz that I made using Python Programming Language. This can only run in the Terminal

This is the Quiz that I made using Python Programming Language. This can only run in the Terminal

YOSHITHA RATHNAYAKE 1 Apr 08, 2022
Automatically load and dump your dataclasses 📂🙋

file dataclasses Installation By default, filedataclasses comes with support for JSON files only. To support other formats like YAML and TOML, filedat

Alon 1 Dec 30, 2021
AndroidEnv is a Python library that exposes an Android device as a Reinforcement Learning (RL) environment.

AndroidEnv is a Python library that exposes an Android device as a Reinforcement Learning (RL) environment.

DeepMind 814 Dec 26, 2022
Python solutions to Codeforces problems

CodeForces This repository is dedicated to my Python solutions for CodeForces problems. Feel free to copy, contribute and/or comment. If you find any

Shukur Sabzaliev 15 Dec 20, 2022
Blender-miHoYo-Shaders - Shaders for Blender attempting to replicate the shading of games developed by miHoYo

Blender-miHoYo-Shaders - Shaders for Blender attempting to replicate the shading of games developed by miHoYo

Matsuri 449 Jan 07, 2023
Python wrapper to different clients to determine how a particular term is used.

Python wrapper to different clients to determine how a particular term is used.

Chris Mungall 3 Oct 24, 2022
pgvector support for Python

pgvector-python pgvector support for Python Great for online recommendations 🎉 Supports Django, SQLAlchemy, Psycopg 2, Psycopg 3, and asyncpg Install

Andrew Kane 37 Dec 20, 2022
Expense Tracker is a very good tool to keep track of your expenseditures and the total money you saved.

Expense Tracker is a very good tool to keep track of your expenseditures and the total money you saved.

Shreejan Dolai 9 Dec 31, 2022
Automate the boilerplate while initializing your Python project

Rubric Automate the boilerplate while initializing your Python project Preface Rubric is an opinionated project initializer for Python. It assum

Redowan Delowar 23 Dec 16, 2022
🐍 This snake helps you reconnect the Web, with RSS feeds!

This snake helps you reconnect the Web, with RSS feeds! RSSerpent is an open-source software that create RSS feeds for websites that do not provide an

211 Dec 08, 2022
An easy way to access to your EPITECH subjects based on the Roslyn's database.

An easy way to access to your EPITECH subjects based on the Roslyn's database.

Mathias 1 Feb 09, 2022
Badge-Link-Creater 'For more beautiful profiles.'

Badge-Link-Creater 'For more beautiful profiles.' Ready Badges Prepares the codes of the previously prepared badges for you. Note Click here for more

Mücahit Gündüz 9 Oct 19, 2022
An Airdrop alternative for cross-platform users only for desktop with Python

PyDrop An Airdrop alternative for cross-platform users only for desktop with Python, -version 1.0 with less effort, just as a practice. ##############

Bernardo Olisan 6 Mar 25, 2022
Hacktoberfest2021 🥳- Contribute Any Pattern In Any Language😎 Every PR will be accepted Pls contribute

✨ Hacktober Fest 2021 ✨ 🙂 All Contributors are requested to star this repo and follow me for a successful merge of pull request. 🙂 👉 Add any patter

Md. Almas Ali 103 Jan 07, 2023