A utility to search, download and process Landsat 8 satellite imagery

Overview

Landsat-util

https://travis-ci.org/developmentseed/landsat-util.svg?branch=master Downloads License

Landsat-util is a command line utility that makes it easy to search, download, and process Landsat imagery.

Docs

For full documentation visit: https://pythonhosted.org/landsat-util/

To run the documentation locally:

$ pip install -r requirements/dev.txt
$ cd docs
$ make html

Recently Added Features

  • Improved pansharpening
  • Use BQA bands for cloud/snow coverage and use in color correction
  • Add support for different NDVI color maps (three included)
  • Add support for image clipping using the new --clip flag

Change Log

See CHANGES.txt.

Comments
  • Searching returns no result

    Searching returns no result

    Hi people!

    landsat is not returning any results. For example:

    $ landsat search -p 228,81 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 23,35 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 226,84 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    

    Any suggestion/pointer to solve this problem? Thanks in advance!

    Matias

    opened by matiaslee 27
  • doesn't work

    doesn't work

    Its probably a simple fix but I'm not an IT guy. I keep getting the following error

    landsat search --cloud 4 --start "january 1 2014" --end "january 10 2014" -p 009,045 Traceback (most recent call last): File "/Users/schull/anaconda/bin/landsat", line 5, in import landsat.landsat File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/landsat.py", line 21, in from image import Process, FileDoesNotExist File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/image.py", line 13, in import rasterio File "/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/init.py", line 8, in from rasterio._base import eval_window, window_shape, window_index ImportError: dlopen(/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so, 2): Library not loaded: libgdal.1.dylib Referenced from: /Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so Reason: image not found

    Any ideas how to fix it?

    opened by bucricket 19
  • Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    I am not having success trying to load a set of specific Landsat-8 scenes from USGS EROS site -- for scenes not available on AWS or Google-- trying this from an OSX machine.

    I get the scene names using '> landsat search' just fine... With '> landsat download', I get "product not available on AWS S3 or Google Storage"... one of these scenes, as an example, is LC81290582016007LGN02.

    When I supply my USGS EROS credentials --username --password (I do have m2m / special inventory access enabled on this account), I get the error message below. Any help greatly appreciated...

    #---- Time spent : 0.75 seconds Traceback (most recent call last): File "/usr/local/bin/landsat", line 11, in sys.exit(main()) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 491, in main exit(*main(args)) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 408, in main files = d.download(args.scenes, bands) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 80, in download files.append(self.usgs_eros(scene, self.download_dir)) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 95, in usgs_eros error_tree = ElementTree.fromstring(str(e.message)) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1311, in XML parser.feed(text) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed self._raiseerror(v) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror raise err xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 171, column 22

    opened by gybely 12
  • returned non-zero exit status 4

    returned non-zero exit status 4

    Hi. Congratulations on this very usefull tool. I'm having trouble processing the images after dowloading them. Here's the console text:

    "C:\Programs\landsat-util-master>python C:\Programs\landsat-util-master\landsat\l andsat.py search --download --imageprocess --cloud 20 --start "january 1 2014" - -end "april 10 2014" pr 204 032 1 items were found { "limit": 100, "results": [ { "cloud": 4.16, "date": "2014-03-19", "path": "204", "row": "032", "sat_type": "L8", "sceneID": "LC82040322014078LGN00", "thumbnail": "http://earthexplorer.usgs.gov/browse/landsat_8/2014/20 4/032/LC82040322014078LGN00.jpg" } ], "status": "SUCCESS", "total": 1, "total_returned": 1 } Starting the download: Skipping existing item: file://C:\Users\Administrator/landsat/zip/LC820403220140 78LGN00.tar.bz 1 images were downloaded c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00 folder cr eated c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp fold er created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/scaled fo lder created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/final fol der created Unzipping LC82040322014078LGN00 - It might take some time Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B4.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B3.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B2.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 15655P x 15862L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B8.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Starting the Min/Max process with designated -percent cut- for band 4 of LC82040 322014078LGN00 ... Finished processing band 4 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 3 of LC82040 322014078LGN00 ... Finished processing band 3 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 2 of LC82040 322014078LGN00 ... Finished processing band 2 of LC82040322014078LGN00 scaling pan to min max with 2% cut for band 4 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 3 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 2 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. Invalid Parameter - -combine Traceback (most recent call last): File "C:\Programs\landsat-util-master\landsat\landsat.py", line 285, in main() File "C:\Programs\landsat-util-master\landsat\landsat.py", line 282, in __main __ main(args) File "C:\Programs\landsat-util-master\landsat\landsat.py", line 236, in main p.full() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 108, in f ull self._combine() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 340, in _ combine subprocess.check_call(argv) File "C:\OSGeo4W64\apps\Python27\lib\subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['convert', '-identify', '-combine', u'c :\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/war p/LC82040322014078LGN00_B4.TIF', u'c:\users\admini~1\appdata\local\temp\1
    \tmpybepa2/LC82040322014078LGN00/warp/LC82040322014078LGN00_B3.TIF', u'c:\users \admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp/LC8204 0322014078LGN00_B2.TIF', u'c:\users\admini~1\appdata\local\temp\1\tmpybep a2/LC82040322014078LGN00/final/rgb-null.TIF']' returned non-zero exit status 4 "

    Installation Issues v0.2 
    opened by vascobnunes 12
  • landsat collection 1

    landsat collection 1

    Hi,

    I'm wondering if you are planning on implementing the new collection 1 information? The ESPA ordering system no longer stores the pre-collection data and so the sceneID information doesn't work anymore. We need the Landsat-product ID now.

    Mitch

    opened by bucricket 11
  • Loosen requirement versions

    Loosen requirement versions

    I essentially widened the allowed versions for all dependencies, taking care of specifying rasterio<1. The current setup has been tested in all Python >= 3.4 versions (see commit logs). I also updated the Dockerfile, hopefully fixing another issue.

    opened by astrojuanlu 10
  • Support for Copernicus Sentinel-2 imaginery

    Support for Copernicus Sentinel-2 imaginery

    This is just a far fetched open question: would be interesting to support the download / processing of imaginery from the Sentinel-2 satellites of the Copernicus mission? A few points of interest:

    The mission

    Multi-spectral data with 13 bands in the visible, near infrared, and short wave infrared part of the spectrum Systematic global coverage of land surfaces from 56° S to 84° N, coastal waters, and all of the Mediterranean Sea Revisiting every 5 days at the Equator under the same viewing conditions Spatial resolution of 10 m, 20 m and 60 m 290 km field of view Free and open data policy

    Imaginery

    Among others, there will be 4 bands at 10m resolution (50% more than Landsat-8): red, green, blue and VNIR. Nicely, there is no need to pansharpen in order to get images at max. resolution.

    bands

    Data access

    Data will be free to access. The good news are that there is an API to perform searches (including geographical bounding, cloud cover, date, etc.) and download quicklooks and products. This API is well documented here. The bad news are that the API requires user registration:

    curl -u username:password "https://scihub.esa.int/dhus/search?q=S2A*&format=json"
    

    Also, the images are distributed in a specific SAFE format which I'm not sure GDAL supports yet (it seems that GDAL supports Sentinel-1 SAFE, but not Sentinel-2 SAFE).


    As you can see the tools are not so ready yet, but I think it could be interesting to have this issue in mind in order to someday extend the functionality of landsat-util, no? The possibility of 50% higher resolution imaginary with 1/3 of the revisit time without the need of pansharpening (which highly increases the resources needed) is very tempting!

    What do you think? =)

    opened by JorgeGT 10
  • update join method for download URL

    update join method for download URL

    replaced join() with "/".join() for building google and Amazon S3 URLs that are compatible across platforms. Remove trailing "/" from S3 and Google base URL in settings.py

    opened by RoboDonut 9
  • Add an option for parallel downloading

    Add an option for parallel downloading

    As I've proposed in Issue #102, these are the modifications in order to allow the use of new command-line flag --use-aria2c true, in order to download the scenes using aria2c's parallel download. This is my first contribution so, in case something looks odd, just let me know, ok? Cheers.

    opened by vbenso 9
  • download band 8 in continuous download/process/pansharpen command

    download band 8 in continuous download/process/pansharpen command

    This command landsat download LC81410412015088LGN00 --bands 432 -p --pansharpen fails because band 8 is not downloaded from aws.

    Have to make sure that band 8 is downloaded if pansharpen flag is used for 2015 and newer imagery.

    opened by scisco 8
  • magick error

    magick error

    After updating Xcode to 4.6.3 and running brew doctor, still i get this error after running the install for landsat-util--has anyone seen this before?

    1 error generated. make[1]: *** [magick/magick_libMagickCore_6_Q16_la-annotate.lo] Error 1 make[1]: *** Waiting for unfinished jobs.... make: *** [install] Error 2

    opened by ebarry 8
  • Coordinates to clip the raster

    Coordinates to clip the raster

    Hi there, I am working on a raster file and want to clip an area from the raster using latitude and longitude. But Landsat API requires coordinates in WGS84 datum. How can I convert these lat and long coordinates into the desired format?. Thanks.

    opened by EhsanAlahi 0
  • Update dependencies for new rasterio version

    Update dependencies for new rasterio version

    Hi,

    Recently rasterio released version 1.0 and is currently at version 1.0.23. That has some important updates and features, and at the moment I cannot install landsat-util because of its rasterio version requirements (currently rasterio~=0.36)

    Would it be possible to update that?

    opened by alessioarena 1
  • Docker search SSL error

    Docker search SSL error

    $ sudo docker run -it -v ~/landsat:/root/landsat developmentseed/landsat-util:latest landsat search --lat 33.9436973 --lon -118.4073388       
    
    /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
      InsecurePlatformWarning
    Traceback (most recent call last):
      File "/usr/local/bin/landsat", line 9, in <module>
        load_entry_point('landsat-util==0.13.1', 'console_scripts', 'landsat')()
      File "/landsat/landsat/landsat.py", line 488, in __main__
        print(main(args))
      File "/landsat/landsat/landsat.py", line 357, in main
        geojson=args.geojson)
      File "/landsat/landsat/search.py", line 96, in search
        r = requests.get('%s?search=%s&limit=%s' % (self.api_url, search_string, limit))
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in get
        return request('get', url, params=params, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in request
        response = session.request(method=method, url=url, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send
        r = adapter.send(request, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 431, in send
        raise SSLError(e, request=request)
    requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error
    
    opened by palewire 1
  • Problem with using landsat-util downloads lansat8 2018 year data

    Problem with using landsat-util downloads lansat8 2018 year data

    When I tried to download Landsat 8 data with year of 2018 , then there was an error LC81210382018100LGN00 is not available on AWS S3 or Google Storage image

    opened by 120534 0
  • Would like to use landsat-util on windows

    Would like to use landsat-util on windows

    I'm using a windows 10 system. Should I use a docker and run it on Hyper-V? Is this the only way? Anyone familiar with the start-up could help me out? Thanks!

    opened by gracie2017 3
Releases(v0.13.1)
  • v0.13.1(Jan 18, 2017)

  • v0.13.0(Mar 25, 2016)

  • v0.12.2(Mar 25, 2016)

  • v0.12.1(Feb 18, 2016)

  • v0.12.0(Feb 18, 2016)

  • v0.10.0(Jan 5, 2016)

    • add support for bare json output
    • faster travis tests
    • add street search
    • apply ndvigrey to process and download
    • fixes #127
    • update commands help file
    • apply pansharpen and ndvi only if -p is used
    • download zip if bands are not specified
    • better handle url joins to a fix error on Windows
    • other small bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.9.1(Jan 5, 2016)

  • v0.8.0(Oct 23, 2015)

    0.8.0 (2015-09-22)

    • Improved docs
    • Add --ndvi flag
    • Handle downloading new bands (10, 11, QA)
    • Improved color correction
    • Remove noise in pansharpened image processing
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(May 29, 2015)

    • New documentation
    • Deployed to readthedocs
    • Automate deployment to pypi
    • Adds docker support
    • skip unzipping if images already unzipped
    • add force-unzip flag
    • fix a bug where multiple downloads was not followed by multiple process #81
    • fix a bug where if scenes was downloaded from google instead of aws, process failed #84
    • download band 8 when pansharpen fixes #73
    Source code(tar.gz)
    Source code(zip)
  • v0.6.3(Apr 29, 2015)

    includes:

    • adjust lower rescaling bound, closes #66 for now
    • better pixel math for #71
    • update destination transform calculation for reprojection in polar regions
    Source code(tar.gz)
    Source code(zip)
  • v0.6.2(Apr 25, 2015)

  • v0.6.1(Apr 23, 2015)

    Includes:

    • Updated tests that run faster
    • New upload command for uploading final image to S3
    • New command for continuous upload, process and upload
    • Updated image processing that produces better color correction
    • Other bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Apr 8, 2015)

  • v0.5.0(Mar 10, 2015)

    Version 0.5.0 includes:

    • Fewer dependencies
    • Uses wheel on OSX
    • New image processing algorithm which is faster and require less storage
    • Updated API
    • Includes longitude latitude search
    • Improved console output
    • Accepts bands combinations for processing images
    • Includes a new faster downloader
    • More comprehensive tests (96% coverage)
    • Added to pypi pip install landsat-util
    • Brew formula removed
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Aug 28, 2014)

Owner
Development Seed
Development Seed
Raster-based Spatial Analysis for Python

🌍 xarray-spatial: Raster-Based Spatial Analysis in Python 📍 Fast, Accurate Python library for Raster Operations ⚡ Extensible with Numba ⏩ Scalable w

makepath 649 Jan 01, 2023
Implemented a Google Maps prototype that provides the shortest route in terms of distance

Implemented a Google Maps prototype that provides the shortest route in terms of distance, the fastest route, the route with the fewest turns, and a scenic route that avoids roads when provided a sou

1 Dec 26, 2021
3D extension built off of shapely to make working with geospatial/trajectory data easier in python.

PyGeoShape 3D extension to shapely and pyproj to make working with geospatial/trajectory data easier in python. Getting Started Installation pip The e

Marc Brittain 5 Dec 27, 2022
Manage your XYZ Hub or HERE Data Hub spaces from Python.

XYZ Spaces for Python Manage your XYZ Hub or HERE Data Hub spaces and Interactive Map Layer from Python. FEATURED IN: Online Python Machine Learning C

HERE Technologies 30 Oct 18, 2022
Specification for storing geospatial vector data (point, line, polygon) in Parquet

GeoParquet About This repository defines how to store geospatial vector data (point, lines, polygons) in Apache Parquet, a popular columnar storage fo

Open Geospatial Consortium 449 Dec 27, 2022
This app displays interesting statistical weather records and trends which can be used in climate related research including study of global warming.

This app displays interesting statistical weather records and trends which can be used in climate related research including study of global warming.

0 Dec 27, 2021
Geospatial web application developed uisng earthengine, geemap, and streamlit.

geospatial-streamlit Geospatial web applications developed uisng earthengine, geemap, and streamlit. App 1 - Land Surface Temperature A simple, code-f

13 Nov 27, 2022
geobeam - adds GIS capabilities to your Apache Beam and Dataflow pipelines.

geobeam adds GIS capabilities to your Apache Beam pipelines. What does geobeam do? geobeam enables you to ingest and analyze massive amounts of geospa

Google Cloud Platform 61 Nov 08, 2022
Streamlit Component for rendering Folium maps

streamlit-folium This Streamlit Component is a work-in-progress to determine what functionality is desirable for a Folium and Streamlit integration. C

Randy Zwitch 224 Dec 30, 2022
Python 台灣行政區地圖 (2021)

Python 台灣行政區地圖 (2021) 以 python 讀取政府開放平台的 ShapeFile 地圖資訊。歡迎引用或是協作 另有縣市資訊、村里資訊與各種行政地圖資訊 例如: 直轄市、縣市界線(TWD97經緯度) 鄉鎮市區界線(TWD97經緯度) | 政府資料開放平臺: https://data

WeselyOng 12 Sep 27, 2022
Wraps GEOS geometry functions in numpy ufuncs.

PyGEOS PyGEOS is a C/Python library with vectorized geometry functions. The geometry operations are done in the open-source geometry library GEOS. PyG

362 Dec 23, 2022
Example of animated maps in matplotlib + geopandas using entire time series of congressional district maps from UCLA archive. rendered, interactive version below

Example of animated maps in matplotlib + geopandas using entire time series of congressional district maps from UCLA archive. rendered, interactive version below

Apoorva Lal 5 May 18, 2022
Deal with Bing Maps Tiles and Pixels / WGS 84 coordinates conversions, and generate grid Shapefiles

PyBingTiles This is a small toolkit in order to deal with Bing Tiles, used i.e. by Facebook for their Data for Good datasets. Install Clone this repos

Shoichi 1 Dec 08, 2021
Python bindings and utilities for GeoJSON

geojson This Python library contains: Functions for encoding and decoding GeoJSON formatted data Classes for all GeoJSON Objects An implementation of

Jazzband 765 Jan 06, 2023
Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

OpenWISP 982 Jan 06, 2023
A simple reverse geocoder that resolves a location to a country

Reverse Geocoder This repository holds a small web service that performs reverse geocoding to determine whether a user specified location is within th

4 Dec 25, 2021
OSMnx: Python for street networks. Retrieve, model, analyze, and visualize street networks and other spatial data from OpenStreetMap.

OSMnx OSMnx is a Python package that lets you download geospatial data from OpenStreetMap and model, project, visualize, and analyze real-world street

Geoff Boeing 4k Jan 08, 2023
Minimum Bounding Box of Geospatial data

BBOX Problem definition: The spatial data users often are required to obtain the coordinates of the minimum bounding box of vector and raster data in

Ali Khosravi Kazazi 1 Sep 08, 2022
Pure python WMS

Ogcserver Python WMS implementation using Mapnik. Depends Mapnik = 0.7.0 (and python bindings) Pillow PasteScript WebOb You will need to install Map

Mapnik 130 Dec 28, 2022
Use Mapbox GL JS to visualize data in a Python Jupyter notebook

Location Data Visualization library for Jupyter Notebooks Library documentation at https://mapbox-mapboxgl-jupyter.readthedocs-hosted.com/en/latest/.

Mapbox 620 Dec 15, 2022