Roadster - Distance to Closest Road Feature Server

Related tags

NetworkingRoadster
Overview

Roadster: Distance to Closest Road Feature Server

Milliarium Aerum, CC-BY-SA Wikipedia

Milliarium Aerum, the zero of all Roman roads

This is a feature server that computes the distance to the closest road as feature for a given coordinate.

It can be used command-line or with the provided Flask microframework server.

Roadster uses .SHP files from the Open Street Map project, as generously provided by Geofabrik GmbH.

It can provide the value of the distance-to-closest-road feature for a given coordinate or compute the value of the feature for all points on a given tile. If a given tile is too big, it can sample the feature and interpolate its value using Ordinary Kriging.

No pandas dependencies.

Installation

python setup.py install

Download the SHP files and put them (in .zip format) in the download/ folder.

Command-line

roadster-one-tile -m {map-prefix} -l {road_layer} -r {road_type} -z -w {tile-width} -h {tile-height} -o output_file -t {brute|kriging} -b {boost} west-north-gps-lat west-north-gps-long west-north-gps-lat east-south-gps-long

roadster-one-coord -m {map-prefix} -l {road_layer} -r {road_type} gps-lat gps-long

Arguments:

  • Map prefix, e.g., malta-latest-free, the .shp.zip extension is added. This file is searched for in download folder. If the parameter contains a file path, the file is used verbatim (and needs to include the extension).
  • Layer that contains the roads by default the first layer is used (which most probably do not contain the roads).
  • Type of roads (e.g., 'primary'), by default use all roads.
  • Set the roads distance to zero (otherwise is set to 1).
  • Tile width and heights, in pixels.
  • Output file, contains the extension, any of the formats understood by scikit-image.
  • Type of computation of the tile, either brute force (distance computed for each pixel) or ordinary Kriging interpolation. If not specified, brute-force is used for tiles up to 128x128 pixels.
  • Boost to the signal, defaults to 1000.0, try smaller numbers is the image is all white.

Finding the layer with roads

Use the provided command list-layers -m {map-prefix}.

API

import roadster

mapdata = roadster.data.load_map('prince-edward-island-latest-free', 11, set(['primary'])) # major roads
featdist = roadster.point.distance(mapdata, (-63.1293, 46.2905)) # long/lat
# ... use feature as needed ...
img = roadster.data.create_image(128, 128)
roadster.tile.plot(img, mapdata, 'brute', (-63.1647, 46.2779), (-63.0914, 46.2329), boost=10)
roadster.data.plot_roads(img, mapdata, (-63.1647, 46.2779), (-63.0914, 46.2329), road_value=1.0)
# ... use image as needed ...
roadster.data.save_image(img, "pei.png")

This computes a feature tile at this location: https://osm.org/go/cgtSeJX

The output tile looks like this:

Tile

Server

The Flask server takes similar parameters as the command-line interfaces. It caches the map data between calls.

flask run &
curl --output pei.png http://localhost:5000/tile/prince-edward-island-latest-free/11/128/128/46.2779/-63.1647/46.2329/-63.0914?zero_roads\&road_type=primary\&boost=10\&type=kriging
curl http://localhost:5000/point/prince-edward-island-latest-free/11/46.2779/-63.1647

Roadmap

  • Handle downloading and caching OSM street data
  • Better sampling for Kriging

Known issues

  • The tool takes the coordinates in lon/lat format as that seems to be the format present in OSM
  • On larger tiles, there are some projection issues that need to be debugged
  • Truly large tiles (2048x2048) need 50Gb or more RAM to process
  • Brute force does not use multiprocessing

Contributing

Pull requests are welcome! If you find Roadster useful, feel free to contribute. This codebase uses black for code formatting, please run it on your files before submitting them.

Acknowledgements

This project started at the #CleanMaltaAI hackathon as part of the Presagis team.

Citing

This code expands on Chapter 10: Other domains Video, geographic information and preferences of The Art of Feature Engineering. If you use this featurizer in your work, please cite:

@book{duboue2020art,
  title={The Art of Feature Engineering: Essentials for Machine Learning},
  author={Duboue, Pablo},
  year={2020},
  month={June},
  isbn={978-1108709385},
  publisher={Cambridge University Press},
  url={http://artoffeatureengineering.com}
}
Owner
Textualization Software Ltd.
Natural Language Generation technology
Textualization Software Ltd.
A simple Tor switcher script switches tor nodes in interval of time

Tor_Switcher A simple Tor switcher script switches tor nodes in interval of time This script will switch tor nodes in every interval of time that you

d4rk sh4d0w 2 Nov 15, 2021
Malcolm is a powerful, easily deployable network traffic analysis tool suite for full packet capture artifacts (PCAP files) and Zeek logs.

Malcolm is a powerful, easily deployable network traffic analysis tool suite for full packet capture artifacts (PCAP files) and Zeek logs.

Cybersecurity and Infrastructure Security Agency 1.3k Jan 08, 2023
Timeouts for popular Python packages

Python Timeouts An unresponsive service can be worse than a down one. It can tie up your entire system if not handled properly. All network requests s

Andrew Kane 11 Nov 22, 2022
Desktop application for checking sites connection in a background mode

Site connectivity checker Desktop application for checking site connection in a background mode by sending ICMP messages. Problem and solution Usually

Karina Singatullina 26 Dec 19, 2022
track IP Address

ipX Table of Contents ipX Welcome Features Uses Author πŸ“ License Welcome find the location of an IP address. Specifically, you can get the following

Ali Shahid 15 Sep 26, 2022
Ping IP addresses and domains in parallel to find the accessible and inaccessible ones.

πŸš€ IPpy Parallel testing of IP addresses and domains in python. Reads IP addresses and domains from a CSV file and gives two lists of accessible and i

Shivam Mathur 54 May 21, 2022
a decompilation of NAP36 the widevine removal software for amz and nf used by p2p groups until it stoped working due to it using expired cdm keys

NAP36 a decompilation of NAP36 the widevine removal software for amz and nf used by p2p groups until it stoped working due to it useing expired cdm ke

9 Aug 29, 2021
A server and client for passing data between computercraft computers/turtles across dimensions or even servers.

ccserver A server and client for passing data between computercraft computers/turtles across dimensions or even servers. pastebin get zUnE5N0v client

1 Jan 22, 2022
sync application configuration and settings across multiple multiplatform devices

sync application configuration and settings across multiple multiplatform devices ✨ Key Features β€’ βš—οΈ Installation β€’ πŸ“‘ How To Use β€’ πŸ€” FAQ β€’ πŸ› οΈ Setu

Souvik 6 Aug 25, 2022
league-connection is a python package to communicate to riot client and league client

league-connection is a python package to communicate to riot client and league client.

Sandbox 1 Sep 13, 2022
Blockchain-Enabled IoT Sensor Framework that uses Augmented Reality and Artificial Intelligence.

Arduino + Raspberry Pi + Unity3D + Cloud + Hyperledger Our Mission: Keep it simple, leave no one behind. Blockchain-Enabled Smart Sensor Framework usi

DappAR 23 Dec 05, 2021
Free,Cross-platform,Single-file mass network protocol server simulator

FaPro Free,Cross-platform,Single-file mass network protocol server simulator δΈ­ζ–‡Readme Description FaPro is a Fake Protocol Server tool, Can easily sta

FOFA Pro 1.4k Jan 06, 2023
πŸ“¨ Share files easily over your local network from the terminal! πŸ“¨

Fileshare πŸ“¨ Share files easily over your local network from the terminal! πŸ“¨ Installation #

Dopevog 11 Sep 10, 2021
Implementing Cisco Support APIs into NetBox

NetBox Cisco Support API Plugin NetBox plugin using Cisco Support APIs to gather EoX and Contract coverage information for Cisco devices. Compatibilit

Timo Reimann 23 Dec 21, 2022
This is an open project to maintain a list of domain names that serve YouTube ads

The YouTube ads blocklist project This is an open project to maintain a list of domain names that serve YouTube ads. The original project only produce

Evan Pratten 574 Dec 30, 2022
DEMO SOCKET AF INET SSL PYTHON

DEMO_SOCKET_AF_INET_SSL_PYTHON Python demo of socket family as AF_INET using TCP with SSL. Compatibility : macOS & GNU/Linux Network Topology style :

Enola 1 Jan 24, 2022
This is a top level socket library, making servers and clients EASY!

quick-net Sockets don't have to be a pain That's the motto this library was built with, and that's exactly what we made! This is a top-level socket li

Nate the great 15 Dec 17, 2021
msgspec is a fast and friendly implementation of the MessagePack protocol for Python 3.8+

msgspec msgspec is a fast and friendly implementation of the MessagePack protocol for Python 3.8+. In addition to serialization/deserializat

Jim Crist-Harif 414 Jan 06, 2023
9SPY: a Windows RAT built in Python using sockets

9SPY πŸ‘β€πŸ—¨ 9SPY is a Windows RAT built in Python using sockets Features Features will be listed here soon, there are currenly 14 Information This is a

doop 12 Dec 01, 2022
πŸ₯‘ A Python ARP and DNS Spoofer CLI and INTERFACE πŸ₯“

NEXTGEN SPOOFER πŸ₯‘ A Python ARP and DNS Spoofer CLI and INTERFACE πŸ₯“ CLI - advanced pentesters INTERFACE - beginners SetUp Make sure you installed P

9 Dec 25, 2022