ASF Sentinel-1 Metadata Download tool

Related tags

Downloaderasfsmd
Overview

ASF Sentinel-1 Metadata Download tool

Copyright: 2021-2022 Antonio Valentino

Small Python tool (asfsmd) that allows to download XML files containing Sentinel-1 products metadata from the ASF archive.

Sentinel-1 products are stored in the ASF arcive as ZIP files that are quite large because they comntain both the products annotations and the binary image data.

The asfsmd tool is able to retrieve only the relatively samll annotation files (in XML format) without downloading the entire ZIP archive.

asfsmd exploits the httpio Python package for reading HTTP resources as random-access file-like objects. In order to do it the remote server must support the Range header.

This approach allows to open the ZIP archive remotely, inspects contents, and download only the pieces of data that are actually necessary to the user.

Performnces of this approach are quite poor but, in the specific case of Sentinel-1 products, the entire process results to be faster than downloading the entire ZIP archive and extracting only annotation files.

Command Line Interface

$ python3 asfsmd.py --help

usage: asfsmd [-h] [--version]
              [--loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
              [-q] [-v] [-d] [-f] [-o OUTDIR] [-u USERNAME] [-p PASSWORD]
              INPUT [INPUT ...]

ASF Sentinel-1 Metadata Download tool. Small Python tool (`asfsmd`) that
allows to download XML files containing Sentinel-1 products metadata from
the ASF archive. Sentinel-1 products are stored in the ASF arcive as ZIP
files that are quite large because they comntain both the products
annotations and the binary image data. The `asfsmd` tool is able to
retrieve only the relatively samll annotation files (in XML format) without
downloading the entire ZIP archive.

positional arguments:
  INPUT                 Sentinel-1 product name(s). If the '-f' flag is set
                        then the argument is interpreted as the filename
                        containing the list of products.

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        logging level (default: WARNING)
  -q, --quiet           suppress standard output messages, only errors are
                        printed to screen
  -v, --verbose         print verbose output messages
  -d, --debug           print debug messages
  -f, --file-list       read the list of products form file. The file is
                        expected to contain one product name per line.
  -o OUTDIR, --outdir OUTDIR
                        path of the output directory (default='.')
  -u USERNAME, --username USERNAME
                        username for ASF authentication. If not provided
                        the tool attempts to retireve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  -p PASSWORD, --password PASSWORD
                        password for ASF authentication. If not provided
                        the tool attempts to retireve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  --block-size BLOCK_SIZE
                        httpio block size in bytes (default: 1024)

License

The asfsmd package is distributed under the MIT License.

You might also like...
Tool to download Netflix in 4k

Netflix-4K-Script Tool to download Netflix in 4k You will need to get a L1 CDM that is whitelsited with Netflix CDM In this script are downgraded

A cli tool to download purchased products from the DLsite.

dlsite-downloader A cli tool to download purchased products from the DLsite. How can I use? This program runs with configurations defined at settings.

Tool To download 4KHDR DV SDR from AppleTV
Tool To download 4KHDR DV SDR from AppleTV

# APPLE-TV 4K Downloader Tool To download 4K HDR DV SDR from AppleTV Hello Fellow Developers/ ! Hi! My name is WVDUMP. I am Leaking the scripts to

The tool allows to download a list of tiktok sounds

dependencies: pip install requests how to use LAUCH THE PROGRAM file (option f)

Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included

WV-AMZN-4K-RIPPER Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included For CDM You can Mail :- [email protected] Note : CDM is not free L

Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie
Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie

vinetrimmer Widevine Decryption Script for Python Modules Amazon Netflix (with [email protected] support) Disney+ VideoLand Boomerang RTE.ie Hello Fellow Dev

A modern CLI to download animes automatically from Twist
A modern CLI to download animes automatically from Twist

Kurby Kurby is a nice and simple CLI that use Twist website, and their huge collection to download animes for free and automatically Animes from Twist

Download courses from khanacademy.org

khan-dl A python script to download courses from Khan Academy using youtube-dl and beautifulsoup4.

1Fichier Download Manager.
1Fichier Download Manager.

1fichier-dl 1Fichier Download Manager. Features ⭐ Manage your downloads ⭐ Bypass time limits Credits All icons, including the app icon, were provided

Comments
  • Consider alternative backends to httpio

    Consider alternative backends to httpio

    The httpio package is very small seems to fit quite well the needs of this project but:

    • it seems to be a not-active project (last commit 2019)
    • it is necessary to have a specialisation of the httpio.SyncHTTPIOFile class to be able to use this package in asfsmd

    Other alternatives exists that could replace httpio as shall be assessed:

    • [X] RemoteZip

      Very interesting, it can be used out of the box and allows to simplify the codebase quite a lot. The performance to get the list of files included in the ZIP archive seems to be better (x2) than the ones that one can get with httpio but the test consisting in the download of annotations of 5 S1 products seems to be 12% slower than httpio.

    • [X] pyremotezip

      Not maintained upstream, Python2 only.

    • [X] smart-open

      Interesting project bur slower (x3-x4) than httpio to retrieve the list of files contained in the ZIP archive.

    • [X] fsspec

      Extremely interesting and feature complete. Used also in Dask. Direct use of the ZipFileSystem is very slow. The use of the HttpFileSystem + zipfile from the Python stdlib seems to work fine, but it is 33% slower than httpio

    opened by avalentino 2
  • Add filter for polarization and additional calibration/noise/rfi downloads

    Add filter for polarization and additional calibration/noise/rfi downloads

    This adds two functions:

    1. Filter the download by polarization (e.g. so you can grab only 3 'vv' files instead of the 6 for vv/vh)
    2. Download the calibration, noise, and RFI xml files

    I also made a separate branch to further filter to a single subswath by checking iw{number}, but these are the additions I've already found useful when testing https://github.com/opera-adt/s1-reader

    opened by scottstanie 1
Releases(v1.3.0)
  • v1.3.0(Dec 18, 2022)

  • v1.2.0(Dec 5, 2022)

    • Refactoring to convert the `asfsmd.py`` module into a package.
    • Support multiple backends for remote file access: httpio, fsspec, remotezip. The httpio based implementation seems to be slightly faster w.r.t. the other ones.
    • Fix issue with the management of default values for the make_patterns function.
    • Improve the management of the download of large files (chunking and progress).
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Dec 3, 2022)

    • Now it is possible to customize the selection of files to be downloaded. Beyond the manifest and the annotation files, now it is also possible to download:

      • calibration annotations
      • noise annotations
      • rfi annotations
      • measurement files

      Moreover now it is possible to select a specific beams or polarizations. Patch developed by @scottstanie and @avalentino.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Dec 3, 2022)

Owner
Antonio Valentino
Antonio Valentino
🔥 A Bot To Telegram For Download High Qulity Videos & Songs From Youtube

🔥 A Bot To Telegram For Download High Qulity Videos & Songs From Youtube 🎗 Fast And Free Bot No Need To Pay ✅ By SL-Alpha-X-Team ⚡

Official Alpha-X-Team Account 7 Aug 31, 2022
Implementation of Cross-category Video Highlight Detection via Set-based Learning (ICCV 2021).

Cross-category Video Highlight Detection via Set-based Learning Introduction This project is an implementation of ``Cross-category Video Highlight Det

Minghao (Alan) Xu 49 Dec 17, 2022
Download minecraft head or skin, allows TLauncher accounts

Minecraft-skin-downloader Download minecraft head or skin, allows TLauncher accounts by BoBkiNN_ Contact: https://vk.com/bobkinnvk Requirements: Modul

3 Apr 03, 2022
A manga download script written in python.

manga-dlp python script to download mangas Description A manga download script written in python. It only supports mangadex.org for now. But support f

Ivan Schaller 15 Nov 28, 2022
PyDownloader - Downloads files and folders at high speed (based on your interent speed).

PyDownloader - Downloads files and folders at high speed (based on your interent speed).

Armen._.G 4 Feb 24, 2022
Code for "Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions"

Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions Codebase for the "Adversarial Motion Priors Make Good Substitutes for Com

Alejandro Escontrela 54 Dec 13, 2022
Archivist - Easily archive 📦 Download folder to Google Drive ☁️

Archivist Script for archiving Download folder by uploading unmodified files to a Google Drive folder. Modified files will remain in the Download fold

Timing Liu 3 Sep 30, 2022
AI Dungeon Catalog Archive Toolkit

AI Dungeon Content Archive Toolkit (AID CAT) AID CAT is a command-line utility that will allow you to download JSON backups of: Your private and publi

Mimi 31 Oct 26, 2022
Music and video downloader, Made with love by Bryan Herrera

Python-Mp3Mp4-Downloader Music and video downloader, Made with love by Bryan Herrera Requirements CHOCOLATELY windows command If your system does not

ርᚱ1ናተᛰ ᚻህᚥተპᚱ 104 Dec 27, 2022
Script for YouTube creators to share dislike count with their viewers.

Stahování disliků z YouTube - milafon Tento skript slouží jako možnost zobrazit divákům počet disliků u YouTube videí. Vyžaduje implementaci ze strany

4 Sep 28, 2022
A fast and small Torrent client made with Python 3.

pico-torrent A fast and small Torrent client made with Python 3. History and context It was programmed by a hacker known as Jazz_Man, around January o

Pindorama 9 Oct 04, 2022
Download your Spotify playlists and songs along with album art and metadata

spotDL Download your Spotify playlists and songs along with album art and metadata The fastest, easiest, and most accurate command-line music download

10.6k Jan 03, 2023
YouTube-Downloader - YouTube Video Downloader made using python

YouTube-Downloader YouTube Videos Downloder made using python.

Shivam 1 Jan 16, 2022
Python based YouTube video Downloader GUI Application.

Youtube video Downloader Python based Youtube video Downloader GUI Application. Installation Python Dependencies Import pytube pip install pytube Im

Naem Azam 1 Jan 03, 2022
The tool allows to download a list of tiktok sounds

dependencies: pip install requests how to use LAUCH THE PROGRAM file (option f)

carpal 3 Jan 21, 2022
This is Yt Downloader. Coded with Python (my first repository)

Get Started Download & install Python first before using this software. Download Python Installing Python and Pytube Library (IMPORTANT) Installing Py

Qi 2 Oct 25, 2021
Libretrofuzz - Fuzzy Retroarch thumbnail downloader

Fuzzy Retroarch thumbnail downloader In Retroarch, when you use the manual scann

8 Nov 26, 2022
Discord Nitro Generator + Checker

Discord Nitro Generator + Checker Usage Download the project files and run main.py You will be prompted with 2 questions the first one being the amoun

509 Jan 02, 2023
Python script to download all images/webms of a 4chan thread

Python3 script to continuously download all images/webms of multiple 4chan thread simultaneously - without installation

Micha Fink 208 Jan 04, 2023
Youtube Downloader is a simple but highly efficient Youtube Video Downloader, made completly using Python

Youtube Downloader is a simple but highly efficient Youtube Video Downloader, made completly using Python

Arsh 2 Nov 26, 2022