Pure Python NetCDF file reader and writer

Related tags

Geolocationpyncf
Overview

Pyncf

Pure Python NetCDF file reading and writing.

Introduction

Inspired by the pyshp library, which provides simple pythonic and dependency free data access to vector data, I wanted to create a library for an increasingly popular file format in the raster part of the GIS world, namely, NetCDF. From landuse to climate data, data sought after by GIS practioners are increasingly often found only in the NetCDF format.

My problem was that existing NetCDF libraries for python all rely on interfacing with underlying C based implementations and can be hard to setup outside the context of a full GDAL stack.

But most of the complexity of the format is in reading the metadata in the header, which makes it easy to implement in python and should not have to suffer from the slowness of python. Reading the actual data, which NetCDF can store a lot of, is where one might argue that a C implementation is needed for reasons of speed. But given that the main purpose of the format data model is to provide efficient access to any part of its vast data without having to read all of it via byte offset pointers, this too can be easily and relatively efficiently implemented in python without significant slowdowns. Besides, in many cases, the main use of NetCDF is not for storing enormously vast raster arrays, but rather for storing multiple relatively small raster arrays on different themes, and of providing variations of these across some dimension, such as time.

All of this makes it feasible and desirable with a pure python implementation for reading and writing NetCDF files, expanding access to the various data sources now using this format to a much broader set of users and applications, especially in portable environments.

Status

Basic metadata and data extraction functional, but has not been tested very extensively, so likely to contain some issues. No file writing implemented yet. Only Classic and 64-bit formats supported so far, though NetCDF-4 should be easy to implement.

Basic usage

Documentation is so far a little sparse, so how about some basic examples.

Basically, you load some data file which allows access to its meta data in the "header" attribute, a dictionary structure based exactly on the format specification, which you will just have to explore for now:

import pyncf
ncfile = pyncf.NetCDF(filepath="somefile.nc")
headerdict = ncfile.header

For more intuitive access to metadata there are also some more specific methods for that, all retrieving dictionaries:

ncfile.get_dimensions()
nc.get_diminfo("time")

ncfile.get_nonrecord_variables()
ncfile.get_record_variables()
nc.get_varinfo("temperature")

When it comes to actual data retrieval, there are two main methods. One for reading a dimension's index values if defined in a variable, and another for retrieving a 2d list of lists of a multidimensional variable's data values, by specifying which two dimensions to get your data for and fixing all remaining dimensions at a certain value:

timelabels = ncfile.read_dimension_values("time")
datamatrix = ncfile.read_2d_data(ydim="latitude", xdim="longitude", time=43)

Author

Karim Bahgat, 2016

Based on the file format description at: http://www.unidata.ucar.edu/software/netcdf/docs/file_format_specifications.html

Changes

0.1.0 (2016-03-26)

  • First alpha version
Owner
Karim Bahgat
Karim Bahgat
ColoringMapAlgorithm-CSP- - Graphical Coloring of Countries with Condition Satisfaction Algorithm

ColoringMapAlgorithm-CSP- Condition Satisfaction Algorithm Output Condition

Kerem TAN 2 Jan 10, 2022
Imperial Valley Geomorphology Map

Roughly maps the extent of basins, basin edges, and mountains in the Imperial Valley by grouping terrain classes from the Iwahashi et al. 2021 California terrian classification model.

0 Dec 13, 2022
A simple python script that, given a location and a date, uses the Nasa Earth API to show a photo taken by the Landsat 8 satellite. The script must be executed on the command-line.

What does it do? Given a location and a date, it uses the Nasa Earth API to show a photo taken by the Landsat 8 satellite. The script must be executed

Caio 42 Nov 26, 2022
Spatial Interpolation Toolbox is a Python-based GUI that is able to interpolate spatial data in vector format.

Spatial Interpolation Toolbox This is the home to Spatial Interpolation Toolbox, a graphical user interface (GUI) for interpolating geographic vector

Michael Ward 2 Nov 01, 2021
GebPy is a Python-based, open source tool for the generation of geological data of minerals, rocks and complete lithological sequences.

GebPy is a Python-based, open source tool for the generation of geological data of minerals, rocks and complete lithological sequences. The data can be generated randomly or with respect to user-defi

Maximilian Beeskow 16 Nov 29, 2022
r.cfdtools 7 Dec 28, 2022
Build, deploy and extract satellite public constellations with one command line.

SatExtractor Build, deploy and extract satellite public constellations with one command line. Table of Contents About The Project Getting Started Stru

Frontier Development Lab 70 Nov 18, 2022
a Geolocator made in python

Geolocator A Geolocator made in python ✨ Features locates ur location using ur ip thats it! 💁‍♀️ How to use first download the locator.py file instal

Portgas D Ace 1 Oct 27, 2021
A Jupyter - Leaflet.js bridge

ipyleaflet A Jupyter / Leaflet bridge enabling interactive maps in the Jupyter notebook. Usage Selecting a basemap for a leaflet map: Loading a geojso

Jupyter Widgets 1.3k Dec 27, 2022
Imports VZD (Latvian State Land Service) open data into postgis enabled database

Python script main.py downloads and imports Latvian addresses into PostgreSQL database. Data contains parishes, counties, cities, towns, and streets.

Kaspars Foigts 7 Oct 26, 2022
🌐 Local tile server for viewing geospatial raster files with ipyleaflet or folium

🌐 Local Tile Server for Geospatial Rasters Need to visualize a rather large (gigabytes) raster you have locally? This is for you. A Flask application

Bane Sullivan 192 Jan 04, 2023
Python script that can be used to generate latitude/longitude coordinates for GOES-16 full-disk extent.

goes-latlon Python script that can be used to generate latitude/longitude coordinates for GOES-16 full-disk extent. 🌎 🛰️ The grid files can be acces

Douglas Uba 3 Apr 06, 2022
Pure Python NetCDF file reader and writer

Pyncf Pure Python NetCDF file reading and writing. Introduction Inspired by the pyshp library, which provides simple pythonic and dependency free data

Karim Bahgat 14 Sep 30, 2022
A short term landscape evolution using a path sampling method to solve water and sediment flow continuity equations and model mass flows over complex topographies.

r.sim.terrain A short-term landscape evolution model that simulates topographic change for both steady state and dynamic flow regimes across a range o

Brendan Harmon 7 Oct 21, 2022
A NASA MEaSUREs project to provide automated, low latency, global glacier flow and elevation change datasets

Notebooks A NASA MEaSUREs project to provide automated, low latency, global glacier flow and elevation change datasets This repository provides tools

NASA Jet Propulsion Laboratory 27 Oct 25, 2022
Simple, concise geographical visualization in Python

Geographic visualizations for HoloViews. Build Status Coverage Latest dev release Latest release Docs What is it? GeoViews is a Python library that ma

HoloViz 445 Jan 02, 2023
Evaluation of file formats in the context of geo-referenced 3D geometries.

Geo-referenced Geometry File Formats Classic geometry file formats as .obj, .off, .ply, .stl or .dae do not support the utilization of coordinate syst

Advanced Information Systems and Technology 11 Mar 02, 2022
3D extension built off of shapely to make working with geospatial/trajectory data easier in python.

PyGeoShape 3D extension to shapely and pyproj to make working with geospatial/trajectory data easier in python. Getting Started Installation pip The e

Marc Brittain 5 Dec 27, 2022
Creates 3D geometries from 2D vector graphics, for use in geodynamic models

geomIO - creating 3D geometries from 2D input This is the Julia and Python version of geomIO, a free open source software to generate 3D volumes and s

3 Feb 01, 2022
Yet Another Time Series Model

Yet Another Timeseries Model (YATSM) master v0.6.x-maintenance Build Coverage Docs DOI | About Yet Another Timeseries Model (YATSM) is a Python packag

Chris Holden 60 Sep 13, 2022