A utility to search, download and process Landsat 8 satellite imagery

Overview

Landsat-util

https://travis-ci.org/developmentseed/landsat-util.svg?branch=master Downloads License

Landsat-util is a command line utility that makes it easy to search, download, and process Landsat imagery.

Docs

For full documentation visit: https://pythonhosted.org/landsat-util/

To run the documentation locally:

$ pip install -r requirements/dev.txt
$ cd docs
$ make html

Recently Added Features

  • Improved pansharpening
  • Use BQA bands for cloud/snow coverage and use in color correction
  • Add support for different NDVI color maps (three included)
  • Add support for image clipping using the new --clip flag

Change Log

See CHANGES.txt.

Comments
  • Searching returns no result

    Searching returns no result

    Hi people!

    landsat is not returning any results. For example:

    $ landsat search -p 228,81 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 23,35 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    ['Search completed!']
    
    $ landsat search -p 226,84 --latest 5
    {
        "limit": 25, 
        "results": [], 
        "status": "SUCCESS", 
        "total": 0, 
        "total_returned": 0
    }
    

    Any suggestion/pointer to solve this problem? Thanks in advance!

    Matias

    opened by matiaslee 27
  • doesn't work

    doesn't work

    Its probably a simple fix but I'm not an IT guy. I keep getting the following error

    landsat search --cloud 4 --start "january 1 2014" --end "january 10 2014" -p 009,045 Traceback (most recent call last): File "/Users/schull/anaconda/bin/landsat", line 5, in import landsat.landsat File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/landsat.py", line 21, in from image import Process, FileDoesNotExist File "/Users/schull/anaconda/lib/python2.7/site-packages/landsat/image.py", line 13, in import rasterio File "/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/init.py", line 8, in from rasterio._base import eval_window, window_shape, window_index ImportError: dlopen(/Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so, 2): Library not loaded: libgdal.1.dylib Referenced from: /Users/schull/anaconda/lib/python2.7/site-packages/rasterio/_base.so Reason: image not found

    Any ideas how to fix it?

    opened by bucricket 19
  • Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    Landsat8 download problem from USGS-EROS for L1C product not available on AWS S3 or Google Storage

    I am not having success trying to load a set of specific Landsat-8 scenes from USGS EROS site -- for scenes not available on AWS or Google-- trying this from an OSX machine.

    I get the scene names using '> landsat search' just fine... With '> landsat download', I get "product not available on AWS S3 or Google Storage"... one of these scenes, as an example, is LC81290582016007LGN02.

    When I supply my USGS EROS credentials --username --password (I do have m2m / special inventory access enabled on this account), I get the error message below. Any help greatly appreciated...

    #---- Time spent : 0.75 seconds Traceback (most recent call last): File "/usr/local/bin/landsat", line 11, in sys.exit(main()) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 491, in main exit(*main(args)) File "/usr/local/lib/python2.7/site-packages/landsat/landsat.py", line 408, in main files = d.download(args.scenes, bands) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 80, in download files.append(self.usgs_eros(scene, self.download_dir)) File "/usr/local/lib/python2.7/site-packages/landsat/downloader.py", line 95, in usgs_eros error_tree = ElementTree.fromstring(str(e.message)) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1311, in XML parser.feed(text) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1653, in feed self._raiseerror(v) File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1517, in _raiseerror raise err xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 171, column 22

    opened by gybely 12
  • returned non-zero exit status 4

    returned non-zero exit status 4

    Hi. Congratulations on this very usefull tool. I'm having trouble processing the images after dowloading them. Here's the console text:

    "C:\Programs\landsat-util-master>python C:\Programs\landsat-util-master\landsat\l andsat.py search --download --imageprocess --cloud 20 --start "january 1 2014" - -end "april 10 2014" pr 204 032 1 items were found { "limit": 100, "results": [ { "cloud": 4.16, "date": "2014-03-19", "path": "204", "row": "032", "sat_type": "L8", "sceneID": "LC82040322014078LGN00", "thumbnail": "http://earthexplorer.usgs.gov/browse/landsat_8/2014/20 4/032/LC82040322014078LGN00.jpg" } ], "status": "SUCCESS", "total": 1, "total_returned": 1 } Starting the download: Skipping existing item: file://C:\Users\Administrator/landsat/zip/LC820403220140 78LGN00.tar.bz 1 images were downloaded c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00 folder cr eated c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp fold er created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/scaled fo lder created c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/final fol der created Unzipping LC82040322014078LGN00 - It might take some time Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B4.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B3.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 7828P x 7932L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B2.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Creating output file that is 15655P x 15862L. Processing input file c:\users\admini~1\appdata\local\temp\1\tmpybepa2/LC8204032 2014078LGN00/LC82040322014078LGN00_B8.TIF. 0...10...20...30...40...50...60...70...80...90...100 - done. Starting the Min/Max process with designated -percent cut- for band 4 of LC82040 322014078LGN00 ... Finished processing band 4 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 3 of LC82040 322014078LGN00 ... Finished processing band 3 of LC82040322014078LGN00 Starting the Min/Max process with designated -percent cut- for band 2 of LC82040 322014078LGN00 ... Finished processing band 2 of LC82040322014078LGN00 scaling pan to min max with 2% cut for band 4 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 3 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. scaling pan to min max with 2% cut for band 2 Input file size is 7828, 7932 0...10...20...30...40...50...60...70...80...90...100 - done. Invalid Parameter - -combine Traceback (most recent call last): File "C:\Programs\landsat-util-master\landsat\landsat.py", line 285, in main() File "C:\Programs\landsat-util-master\landsat\landsat.py", line 282, in __main __ main(args) File "C:\Programs\landsat-util-master\landsat\landsat.py", line 236, in main p.full() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 108, in f ull self._combine() File "C:\Programs\landsat-util-master\landsat\image_helper.py", line 340, in _ combine subprocess.check_call(argv) File "C:\OSGeo4W64\apps\Python27\lib\subprocess.py", line 542, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['convert', '-identify', '-combine', u'c :\users\admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/war p/LC82040322014078LGN00_B4.TIF', u'c:\users\admini~1\appdata\local\temp\1
    \tmpybepa2/LC82040322014078LGN00/warp/LC82040322014078LGN00_B3.TIF', u'c:\users \admini~1\appdata\local\temp\1\tmpybepa2/LC82040322014078LGN00/warp/LC8204 0322014078LGN00_B2.TIF', u'c:\users\admini~1\appdata\local\temp\1\tmpybep a2/LC82040322014078LGN00/final/rgb-null.TIF']' returned non-zero exit status 4 "

    Installation Issues v0.2 
    opened by vascobnunes 12
  • landsat collection 1

    landsat collection 1

    Hi,

    I'm wondering if you are planning on implementing the new collection 1 information? The ESPA ordering system no longer stores the pre-collection data and so the sceneID information doesn't work anymore. We need the Landsat-product ID now.

    Mitch

    opened by bucricket 11
  • Loosen requirement versions

    Loosen requirement versions

    I essentially widened the allowed versions for all dependencies, taking care of specifying rasterio<1. The current setup has been tested in all Python >= 3.4 versions (see commit logs). I also updated the Dockerfile, hopefully fixing another issue.

    opened by astrojuanlu 10
  • Support for Copernicus Sentinel-2 imaginery

    Support for Copernicus Sentinel-2 imaginery

    This is just a far fetched open question: would be interesting to support the download / processing of imaginery from the Sentinel-2 satellites of the Copernicus mission? A few points of interest:

    The mission

    Multi-spectral data with 13 bands in the visible, near infrared, and short wave infrared part of the spectrum Systematic global coverage of land surfaces from 56° S to 84° N, coastal waters, and all of the Mediterranean Sea Revisiting every 5 days at the Equator under the same viewing conditions Spatial resolution of 10 m, 20 m and 60 m 290 km field of view Free and open data policy

    Imaginery

    Among others, there will be 4 bands at 10m resolution (50% more than Landsat-8): red, green, blue and VNIR. Nicely, there is no need to pansharpen in order to get images at max. resolution.

    bands

    Data access

    Data will be free to access. The good news are that there is an API to perform searches (including geographical bounding, cloud cover, date, etc.) and download quicklooks and products. This API is well documented here. The bad news are that the API requires user registration:

    curl -u username:password "https://scihub.esa.int/dhus/search?q=S2A*&format=json"
    

    Also, the images are distributed in a specific SAFE format which I'm not sure GDAL supports yet (it seems that GDAL supports Sentinel-1 SAFE, but not Sentinel-2 SAFE).


    As you can see the tools are not so ready yet, but I think it could be interesting to have this issue in mind in order to someday extend the functionality of landsat-util, no? The possibility of 50% higher resolution imaginary with 1/3 of the revisit time without the need of pansharpening (which highly increases the resources needed) is very tempting!

    What do you think? =)

    opened by JorgeGT 10
  • update join method for download URL

    update join method for download URL

    replaced join() with "/".join() for building google and Amazon S3 URLs that are compatible across platforms. Remove trailing "/" from S3 and Google base URL in settings.py

    opened by RoboDonut 9
  • Add an option for parallel downloading

    Add an option for parallel downloading

    As I've proposed in Issue #102, these are the modifications in order to allow the use of new command-line flag --use-aria2c true, in order to download the scenes using aria2c's parallel download. This is my first contribution so, in case something looks odd, just let me know, ok? Cheers.

    opened by vbenso 9
  • download band 8 in continuous download/process/pansharpen command

    download band 8 in continuous download/process/pansharpen command

    This command landsat download LC81410412015088LGN00 --bands 432 -p --pansharpen fails because band 8 is not downloaded from aws.

    Have to make sure that band 8 is downloaded if pansharpen flag is used for 2015 and newer imagery.

    opened by scisco 8
  • magick error

    magick error

    After updating Xcode to 4.6.3 and running brew doctor, still i get this error after running the install for landsat-util--has anyone seen this before?

    1 error generated. make[1]: *** [magick/magick_libMagickCore_6_Q16_la-annotate.lo] Error 1 make[1]: *** Waiting for unfinished jobs.... make: *** [install] Error 2

    opened by ebarry 8
  • Coordinates to clip the raster

    Coordinates to clip the raster

    Hi there, I am working on a raster file and want to clip an area from the raster using latitude and longitude. But Landsat API requires coordinates in WGS84 datum. How can I convert these lat and long coordinates into the desired format?. Thanks.

    opened by EhsanAlahi 0
  • Update dependencies for new rasterio version

    Update dependencies for new rasterio version

    Hi,

    Recently rasterio released version 1.0 and is currently at version 1.0.23. That has some important updates and features, and at the moment I cannot install landsat-util because of its rasterio version requirements (currently rasterio~=0.36)

    Would it be possible to update that?

    opened by alessioarena 1
  • Docker search SSL error

    Docker search SSL error

    $ sudo docker run -it -v ~/landsat:/root/landsat developmentseed/landsat-util:latest landsat search --lat 33.9436973 --lon -118.4073388       
    
    /usr/local/lib/python2.7/dist-packages/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
      InsecurePlatformWarning
    Traceback (most recent call last):
      File "/usr/local/bin/landsat", line 9, in <module>
        load_entry_point('landsat-util==0.13.1', 'console_scripts', 'landsat')()
      File "/landsat/landsat/landsat.py", line 488, in __main__
        print(main(args))
      File "/landsat/landsat/landsat.py", line 357, in main
        geojson=args.geojson)
      File "/landsat/landsat/search.py", line 96, in search
        r = requests.get('%s?search=%s&limit=%s' % (self.api_url, search_string, limit))
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 69, in get
        return request('get', url, params=params, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 50, in request
        response = session.request(method=method, url=url, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 465, in request
        resp = self.send(prep, **send_kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 573, in send
        r = adapter.send(request, **kwargs)
      File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 431, in send
        raise SSLError(e, request=request)
    requests.exceptions.SSLError: [Errno 1] _ssl.c:510: error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error
    
    opened by palewire 1
  • Problem with using landsat-util downloads lansat8 2018 year data

    Problem with using landsat-util downloads lansat8 2018 year data

    When I tried to download Landsat 8 data with year of 2018 , then there was an error LC81210382018100LGN00 is not available on AWS S3 or Google Storage image

    opened by 120534 0
  • Would like to use landsat-util on windows

    Would like to use landsat-util on windows

    I'm using a windows 10 system. Should I use a docker and run it on Hyper-V? Is this the only way? Anyone familiar with the start-up could help me out? Thanks!

    opened by gracie2017 3
Releases(v0.13.1)
  • v0.13.1(Jan 18, 2017)

  • v0.13.0(Mar 25, 2016)

  • v0.12.2(Mar 25, 2016)

  • v0.12.1(Feb 18, 2016)

  • v0.12.0(Feb 18, 2016)

  • v0.10.0(Jan 5, 2016)

    • add support for bare json output
    • faster travis tests
    • add street search
    • apply ndvigrey to process and download
    • fixes #127
    • update commands help file
    • apply pansharpen and ndvi only if -p is used
    • download zip if bands are not specified
    • better handle url joins to a fix error on Windows
    • other small bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.9.1(Jan 5, 2016)

  • v0.8.0(Oct 23, 2015)

    0.8.0 (2015-09-22)

    • Improved docs
    • Add --ndvi flag
    • Handle downloading new bands (10, 11, QA)
    • Improved color correction
    • Remove noise in pansharpened image processing
    Source code(tar.gz)
    Source code(zip)
  • v0.7.0(May 29, 2015)

    • New documentation
    • Deployed to readthedocs
    • Automate deployment to pypi
    • Adds docker support
    • skip unzipping if images already unzipped
    • add force-unzip flag
    • fix a bug where multiple downloads was not followed by multiple process #81
    • fix a bug where if scenes was downloaded from google instead of aws, process failed #84
    • download band 8 when pansharpen fixes #73
    Source code(tar.gz)
    Source code(zip)
  • v0.6.3(Apr 29, 2015)

    includes:

    • adjust lower rescaling bound, closes #66 for now
    • better pixel math for #71
    • update destination transform calculation for reprojection in polar regions
    Source code(tar.gz)
    Source code(zip)
  • v0.6.2(Apr 25, 2015)

  • v0.6.1(Apr 23, 2015)

    Includes:

    • Updated tests that run faster
    • New upload command for uploading final image to S3
    • New command for continuous upload, process and upload
    • Updated image processing that produces better color correction
    • Other bug fixes
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(Apr 8, 2015)

  • v0.5.0(Mar 10, 2015)

    Version 0.5.0 includes:

    • Fewer dependencies
    • Uses wheel on OSX
    • New image processing algorithm which is faster and require less storage
    • Updated API
    • Includes longitude latitude search
    • Improved console output
    • Accepts bands combinations for processing images
    • Includes a new faster downloader
    • More comprehensive tests (96% coverage)
    • Added to pypi pip install landsat-util
    • Brew formula removed
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Aug 28, 2014)

Owner
Development Seed
Development Seed
When traveling in the backcountry during winter time, updating yourself on current and recent weather data is important to understand likely avalanche danger.

Weather Data When traveling in the backcountry during winter time, updating yourself on current and recent weather data is important to understand lik

Trevor Allen 0 Jan 02, 2022
A Python framework for building geospatial web-applications

Hey there, this is Greppo... A Python framework for building geospatial web-applications. Greppo is an open-source Python framework that makes it easy

Greppo 304 Dec 27, 2022
Imports VZD (Latvian State Land Service) open data into postgis enabled database

Python script main.py downloads and imports Latvian addresses into PostgreSQL database. Data contains parishes, counties, cities, towns, and streets.

Kaspars Foigts 7 Oct 26, 2022
Tool to suck data from ArcGIS Server and spit it into PostgreSQL

chupaESRI About ChupaESRI is a Python module/command line tool to extract features from ArcGIS Server map services. Name? Think "chupacabra" or "Chupa

John Reiser 34 Dec 04, 2022
Solving the Traveling Salesman Problem using Self-Organizing Maps

Solving the Traveling Salesman Problem using Self-Organizing Maps This repository contains an implementation of a Self Organizing Map that can be used

Diego Vicente 3.1k Dec 31, 2022
Python library to visualize circular plasmid maps

Plasmidviewer Plasmidviewer is a Python library to visualize plasmid maps from GenBank. This library provides only the function to visualize circular

Mori Hideto 9 Dec 04, 2022
Deal with Bing Maps Tiles and Pixels / WGS 84 coordinates conversions, and generate grid Shapefiles

PyBingTiles This is a small toolkit in order to deal with Bing Tiles, used i.e. by Facebook for their Data for Good datasets. Install Clone this repos

Shoichi 1 Dec 08, 2021
Get Landsat surface reflectance time-series from google earth engine

geextract Google Earth Engine data extraction tool. Quickly obtain Landsat multispectral time-series for exploratory analysis and algorithm testing On

Loïc Dutrieux 50 Dec 15, 2022
Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

Geographic add-ons for Django REST Framework. Maintained by the OpenWISP Project.

OpenWISP 982 Jan 06, 2023
Expose a GDAL file as a HTTP accessible on-the-fly COG

cogserver Expose any GDAL recognized raster file as a HTTP accessible on-the-fly COG (Cloud Optimized GeoTIFF) The on-the-fly COG file is not material

Even Rouault 73 Aug 04, 2022
pure-Python (Numpy optional) 3D coordinate conversions for geospace ecef enu eci

Python 3-D coordinate conversions Pure Python (no prerequistes beyond Python itself) 3-D geographic coordinate conversions and geodesy. API similar to

Geospace code 292 Dec 29, 2022
Computer Vision in Python

Mahotas Python Computer Vision Library Mahotas is a library of fast computer vision algorithms (all implemented in C++ for speed) operating over numpy

Luis Pedro Coelho 792 Dec 20, 2022
Track International space station with python

NASA-ISS-tracker Track International space station with python Modules import json import turtle import urllib.request import time import webbrowser i

Nikhil Yadav 8 Aug 12, 2021
Geocode rows in a SQLite database table

Geocode rows in a SQLite database table

Chris Amico 225 Dec 08, 2022
Python library to decrypt Airtag reports, as well as a InfluxDB/Grafana self-hosted dashboard example

Openhaystack-python This python daemon will allow you to gather your Openhaystack-based airtag reports and display them on a Grafana dashboard. You ca

Bezmenov Denys 19 Jan 03, 2023
Program that shows all the details of the given IP address. Build with Python and ipinfo.io API

ip-details This is a program that shows all the details of the given IP address. Build with Python and ipinfo.io API Usage To use this program, run th

4 Mar 01, 2022
Create Siege configuration files from Cloud Optimized GeoTIFF.

cogeo-siege Documentation: Source Code: https://github.com/developmentseed/cogeo-siege Description Create siege configuration files from Cloud Optimiz

Development Seed 3 Dec 01, 2022
Advanced raster and geometry manipulations

buzzard In a nutshell, the buzzard library provides powerful abstractions to manipulate together images and geometries that come from different kind o

Earthcube Lab 30 Jun 20, 2022
3D extension built off of shapely to make working with geospatial/trajectory data easier in python.

PyGeoShape 3D extension to shapely and pyproj to make working with geospatial/trajectory data easier in python. Getting Started Installation pip The e

Marc Brittain 5 Dec 27, 2022
Replace MSFS2020's bing map to google map

English verison here 中文 免责声明 本教程提到的方法仅用于研究和学习用途。我不对使用、拓展该教程及方法所造成的任何法律责任和损失负责。 背景 微软模拟飞行2020的地景使用了Bing的卫星地图,然而卫星地图比较老旧,很多地区都是几年前的图设置直接是没有的。这种现象在全球不同地区

hesicong 272 Dec 24, 2022