Roadster - Distance to Closest Road Feature Server

Related tags

NetworkingRoadster
Overview

Roadster: Distance to Closest Road Feature Server

Milliarium Aerum, CC-BY-SA Wikipedia

Milliarium Aerum, the zero of all Roman roads

This is a feature server that computes the distance to the closest road as feature for a given coordinate.

It can be used command-line or with the provided Flask microframework server.

Roadster uses .SHP files from the Open Street Map project, as generously provided by Geofabrik GmbH.

It can provide the value of the distance-to-closest-road feature for a given coordinate or compute the value of the feature for all points on a given tile. If a given tile is too big, it can sample the feature and interpolate its value using Ordinary Kriging.

No pandas dependencies.

Installation

python setup.py install

Download the SHP files and put them (in .zip format) in the download/ folder.

Command-line

roadster-one-tile -m {map-prefix} -l {road_layer} -r {road_type} -z -w {tile-width} -h {tile-height} -o output_file -t {brute|kriging} -b {boost} west-north-gps-lat west-north-gps-long west-north-gps-lat east-south-gps-long

roadster-one-coord -m {map-prefix} -l {road_layer} -r {road_type} gps-lat gps-long

Arguments:

  • Map prefix, e.g., malta-latest-free, the .shp.zip extension is added. This file is searched for in download folder. If the parameter contains a file path, the file is used verbatim (and needs to include the extension).
  • Layer that contains the roads by default the first layer is used (which most probably do not contain the roads).
  • Type of roads (e.g., 'primary'), by default use all roads.
  • Set the roads distance to zero (otherwise is set to 1).
  • Tile width and heights, in pixels.
  • Output file, contains the extension, any of the formats understood by scikit-image.
  • Type of computation of the tile, either brute force (distance computed for each pixel) or ordinary Kriging interpolation. If not specified, brute-force is used for tiles up to 128x128 pixels.
  • Boost to the signal, defaults to 1000.0, try smaller numbers is the image is all white.

Finding the layer with roads

Use the provided command list-layers -m {map-prefix}.

API

import roadster

mapdata = roadster.data.load_map('prince-edward-island-latest-free', 11, set(['primary'])) # major roads
featdist = roadster.point.distance(mapdata, (-63.1293, 46.2905)) # long/lat
# ... use feature as needed ...
img = roadster.data.create_image(128, 128)
roadster.tile.plot(img, mapdata, 'brute', (-63.1647, 46.2779), (-63.0914, 46.2329), boost=10)
roadster.data.plot_roads(img, mapdata, (-63.1647, 46.2779), (-63.0914, 46.2329), road_value=1.0)
# ... use image as needed ...
roadster.data.save_image(img, "pei.png")

This computes a feature tile at this location: https://osm.org/go/cgtSeJX

The output tile looks like this:

Tile

Server

The Flask server takes similar parameters as the command-line interfaces. It caches the map data between calls.

flask run &
curl --output pei.png http://localhost:5000/tile/prince-edward-island-latest-free/11/128/128/46.2779/-63.1647/46.2329/-63.0914?zero_roads\&road_type=primary\&boost=10\&type=kriging
curl http://localhost:5000/point/prince-edward-island-latest-free/11/46.2779/-63.1647

Roadmap

  • Handle downloading and caching OSM street data
  • Better sampling for Kriging

Known issues

  • The tool takes the coordinates in lon/lat format as that seems to be the format present in OSM
  • On larger tiles, there are some projection issues that need to be debugged
  • Truly large tiles (2048x2048) need 50Gb or more RAM to process
  • Brute force does not use multiprocessing

Contributing

Pull requests are welcome! If you find Roadster useful, feel free to contribute. This codebase uses black for code formatting, please run it on your files before submitting them.

Acknowledgements

This project started at the #CleanMaltaAI hackathon as part of the Presagis team.

Citing

This code expands on Chapter 10: Other domains Video, geographic information and preferences of The Art of Feature Engineering. If you use this featurizer in your work, please cite:

@book{duboue2020art,
  title={The Art of Feature Engineering: Essentials for Machine Learning},
  author={Duboue, Pablo},
  year={2020},
  month={June},
  isbn={978-1108709385},
  publisher={Cambridge University Press},
  url={http://artoffeatureengineering.com}
}
Owner
Textualization Software Ltd.
Natural Language Generation technology
Textualization Software Ltd.
A simple chat room using socket and threading for handle multiple connections.

• Socket Chat Room was a little project for socket study. It works with a server handling the incoming connections from the clients. Clients send encoded messages while waiting for others clients mes

Guilherme de Oliveira 2 Mar 03, 2022
Herramienta para transferir eventos de Shadowserver REST API hacia Azure Blob Storage.

Herramienta para transferir eventos de Shadowserver REST API hacia Azure Blob Storage.

CSIRT-RD 1 Feb 04, 2022
Real-time text-editor using python tcp socket

Real-time text-editor using python tcp socket This project does not need any external libraries so you don't need to use virtual environments. All you

MatiYo 3 Aug 05, 2022
FIRM-AFL is the first high-throughput greybox fuzzer for IoT firmware.

FIRM-AFL is the first high-throughput greybox fuzzer for IoT firmware. FIRM-AFL addresses two fundamental problems in IoT fuzzing

356 Dec 23, 2022
The AKS cluster provisioner provisions AKS clusters :-)

Overview The AKS cluster provisioner provisions AKS clusters :-) It uses the Azure CLI to configure VNet and subnets before creating the cluster itsel

Gigi Sayfan 1 Nov 10, 2021
Simple python script for automated network scans with random name generator(useful for CTF boxes).

📄 Automated NMAP script Description Simple python script for automated network scans with random name generator(useful for CTF boxes). Requirements 1

Dhmos Funk 2 Oct 29, 2021
Very simple FTP client, sync folder to FTP server, use python, opensource

ftp-sync-python Opensource, A way to safe your data, avoid lost data by Virus, Randsomware Some functions: Upload a folder automatically to FTP server

4 Sep 13, 2022
Transfer files to and from a Windows host via ICMP in restricted network environments.

ICMP-TransferTools ICMP-TransferTools is a set of scripts designed to move files to and from Windows hosts in restricted network environments. This is

icyguider 269 Dec 20, 2022
Start a simple TCP Listener on a specified IP Address and Port Number and receive incoming connections.

About Start a simple TCP Listener on a specified IP Address and Port Number and receive incoming connections. Download Clone using git in terminal(git

AgentGeneric 5 Feb 24, 2022
SMS Based Headless Browsing

Browse the internet without a network connection - Submission for ConUHacks VI

Zafir Khalid 2 Feb 07, 2022
A pure-Python KSUID implementation

Svix - Webhooks as a service Svix-KSUID This library is inspired by Segment's KSUID implementation: https://github.com/segmentio/ksuid What is a ksuid

Svix 83 Dec 16, 2022
Very simple and tiny file sharing service written in python

Simple File sharing tool Sharing client usage (You will need to use python3 for linux) main.py --send -f file/path Optionnal flags : pwd : set a passw

2 Jul 26, 2022
Simple Port Scanner With Socket Module In Python 3x

PortScanner Simple Port Scanner With Socket Module In Python 3x How To Install Requirements Of This Port Scanner sudo apt install python3;sudo apt ins

1 Nov 23, 2021
Module for convenient work with TCP sockets.

m_socket-py Module for convenient work with TCP sockets. Contributing Pool Request is supported! Ask questions in the Issues section. License Copyrigh

Egor Arskiy 5 Mar 09, 2022
Distribute a portion of your yield to other addresses 💙

YSHARE Distribute a portion of your yield to other addresses. How does it work Desposit your yToken or tokens into this contract Set the benificiaries

11 Nov 24, 2021
This Tool can help enginners and biggener in network, the tool help you to find of any ip with subnet mask that can calucate them and show you ( Availble IP's , Subnet Mask, Network-ID, Broadcast-ID )

This Tool can help enginners and biggener in network, the tool help you to find of any ip with subnet mask that can calucate them and show you ( Availble IP's , Subnet Mask, Network-ID, Broadcast-ID

12 Dec 13, 2022
Slowloris is basically an HTTP Denial of Service attack that affects threaded servers.

slowrise-ddos-tool What is Slowloris? Slowloris is basically an HTTP Denial of S

DEMON cat 4 Jun 19, 2022
Dnspython is a DNS toolkit for Python.

dnspython is a DNS toolkit for Python. It supports almost all record types. It can be used for queries, zone transfers, and dynamic updates. It supports TSIG authenticated messages and EDNS0.

Bob Halley 2.1k Jan 06, 2023
Tool written on Python that locate all up host on your subnet

HOSTSCAN Easy to use command line network host scanner. From noob to noobs. Dependencies Nmap 7.92 or superior Python 3.9 or superior All requirements

NexCreep 4 Feb 27, 2022
Get Your Localhost Online - Ngrok Alternative

Get Your Localhost Online - Ngrok Alternative

Azimjon Pulatov 442 Jan 04, 2023