ASF Sentinel-1 Metadata Download tool

Related tags

Downloaderasfsmd
Overview

ASF Sentinel-1 Metadata Download tool

Copyright: 2021-2022 Antonio Valentino

Small Python tool (asfsmd) that allows to download XML files containing Sentinel-1 products metadata from the ASF archive.

Sentinel-1 products are stored in the ASF arcive as ZIP files that are quite large because they comntain both the products annotations and the binary image data.

The asfsmd tool is able to retrieve only the relatively samll annotation files (in XML format) without downloading the entire ZIP archive.

asfsmd exploits the httpio Python package for reading HTTP resources as random-access file-like objects. In order to do it the remote server must support the Range header.

This approach allows to open the ZIP archive remotely, inspects contents, and download only the pieces of data that are actually necessary to the user.

Performnces of this approach are quite poor but, in the specific case of Sentinel-1 products, the entire process results to be faster than downloading the entire ZIP archive and extracting only annotation files.

Command Line Interface

$ python3 asfsmd.py --help

usage: asfsmd [-h] [--version]
              [--loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
              [-q] [-v] [-d] [-f] [-o OUTDIR] [-u USERNAME] [-p PASSWORD]
              INPUT [INPUT ...]

ASF Sentinel-1 Metadata Download tool. Small Python tool (`asfsmd`) that
allows to download XML files containing Sentinel-1 products metadata from
the ASF archive. Sentinel-1 products are stored in the ASF arcive as ZIP
files that are quite large because they comntain both the products
annotations and the binary image data. The `asfsmd` tool is able to
retrieve only the relatively samll annotation files (in XML format) without
downloading the entire ZIP archive.

positional arguments:
  INPUT                 Sentinel-1 product name(s). If the '-f' flag is set
                        then the argument is interpreted as the filename
                        containing the list of products.

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --loglevel {DEBUG,INFO,WARNING,ERROR,CRITICAL}
                        logging level (default: WARNING)
  -q, --quiet           suppress standard output messages, only errors are
                        printed to screen
  -v, --verbose         print verbose output messages
  -d, --debug           print debug messages
  -f, --file-list       read the list of products form file. The file is
                        expected to contain one product name per line.
  -o OUTDIR, --outdir OUTDIR
                        path of the output directory (default='.')
  -u USERNAME, --username USERNAME
                        username for ASF authentication. If not provided
                        the tool attempts to retireve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  -p PASSWORD, --password PASSWORD
                        password for ASF authentication. If not provided
                        the tool attempts to retireve the authentication
                        parameters for the user's '.netrc' file looking
                        for the host 'urs.earthdata.nasa.gov'
  --block-size BLOCK_SIZE
                        httpio block size in bytes (default: 1024)

License

The asfsmd package is distributed under the MIT License.

You might also like...
Tool to download Netflix in 4k

Netflix-4K-Script Tool to download Netflix in 4k You will need to get a L1 CDM that is whitelsited with Netflix CDM In this script are downgraded

A cli tool to download purchased products from the DLsite.

dlsite-downloader A cli tool to download purchased products from the DLsite. How can I use? This program runs with configurations defined at settings.

Tool To download 4KHDR DV SDR from AppleTV
Tool To download 4KHDR DV SDR from AppleTV

# APPLE-TV 4K Downloader Tool To download 4K HDR DV SDR from AppleTV Hello Fellow Developers/ ! Hi! My name is WVDUMP. I am Leaking the scripts to

The tool allows to download a list of tiktok sounds

dependencies: pip install requests how to use LAUCH THE PROGRAM file (option f)

Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included

WV-AMZN-4K-RIPPER Tool To download Amazon 4k SDR HDR 1080, CDM IS Not Included For CDM You can Mail :- [email protected] Note : CDM is not free L

Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie
Tool To download - Amazon - Netflix- Disney+ - VideoLand - Boomerang - RTE.ie

vinetrimmer Widevine Decryption Script for Python Modules Amazon Netflix (with [email protected] support) Disney+ VideoLand Boomerang RTE.ie Hello Fellow Dev

A modern CLI to download animes automatically from Twist
A modern CLI to download animes automatically from Twist

Kurby Kurby is a nice and simple CLI that use Twist website, and their huge collection to download animes for free and automatically Animes from Twist

Download courses from khanacademy.org

khan-dl A python script to download courses from Khan Academy using youtube-dl and beautifulsoup4.

1Fichier Download Manager.
1Fichier Download Manager.

1fichier-dl 1Fichier Download Manager. Features ⭐ Manage your downloads ⭐ Bypass time limits Credits All icons, including the app icon, were provided

Comments
  • Consider alternative backends to httpio

    Consider alternative backends to httpio

    The httpio package is very small seems to fit quite well the needs of this project but:

    • it seems to be a not-active project (last commit 2019)
    • it is necessary to have a specialisation of the httpio.SyncHTTPIOFile class to be able to use this package in asfsmd

    Other alternatives exists that could replace httpio as shall be assessed:

    • [X] RemoteZip

      Very interesting, it can be used out of the box and allows to simplify the codebase quite a lot. The performance to get the list of files included in the ZIP archive seems to be better (x2) than the ones that one can get with httpio but the test consisting in the download of annotations of 5 S1 products seems to be 12% slower than httpio.

    • [X] pyremotezip

      Not maintained upstream, Python2 only.

    • [X] smart-open

      Interesting project bur slower (x3-x4) than httpio to retrieve the list of files contained in the ZIP archive.

    • [X] fsspec

      Extremely interesting and feature complete. Used also in Dask. Direct use of the ZipFileSystem is very slow. The use of the HttpFileSystem + zipfile from the Python stdlib seems to work fine, but it is 33% slower than httpio

    opened by avalentino 2
  • Add filter for polarization and additional calibration/noise/rfi downloads

    Add filter for polarization and additional calibration/noise/rfi downloads

    This adds two functions:

    1. Filter the download by polarization (e.g. so you can grab only 3 'vv' files instead of the 6 for vv/vh)
    2. Download the calibration, noise, and RFI xml files

    I also made a separate branch to further filter to a single subswath by checking iw{number}, but these are the additions I've already found useful when testing https://github.com/opera-adt/s1-reader

    opened by scottstanie 1
Releases(v1.3.0)
  • v1.3.0(Dec 18, 2022)

  • v1.2.0(Dec 5, 2022)

    • Refactoring to convert the `asfsmd.py`` module into a package.
    • Support multiple backends for remote file access: httpio, fsspec, remotezip. The httpio based implementation seems to be slightly faster w.r.t. the other ones.
    • Fix issue with the management of default values for the make_patterns function.
    • Improve the management of the download of large files (chunking and progress).
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Dec 3, 2022)

    • Now it is possible to customize the selection of files to be downloaded. Beyond the manifest and the annotation files, now it is also possible to download:

      • calibration annotations
      • noise annotations
      • rfi annotations
      • measurement files

      Moreover now it is possible to select a specific beams or polarizations. Patch developed by @scottstanie and @avalentino.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(Dec 3, 2022)

Owner
Antonio Valentino
Antonio Valentino
Ripurei is a free-to-use osu! replay downloader, that can be configured to download from any osu! server.

Ripurei Ripurei is a fully functional osu! replay downloader, fully capable of downloading from almost any osu! server. Functionality Timeline ✔️ Able

Thomas 0 Feb 11, 2022
A simple Python program which uses youtube-dl for downloading YouTube videos as mp3 files.

yt-mp3 converter This is a simple Python program which uses youtube-dl for downloading YouTube videos as mp3 files. This program is for you if you are

nostalgicnerdpenguin 1 Oct 24, 2021
This repository contains code for a youtube-dl GUI written in PyQt.

youtube-dl-GUI This repository contains code for a youtube-dl GUI written in PyQt. It is based on youtube-dl which is a Video downloading script maint

M.Yasoob Ullah Khalid ☺ 191 Jan 02, 2023
SubGrab is a utility that allows you to automate subtitles downloading for your media files.

SubGrab - Command-line Subtitles Downloader: A utility which provides an ease for automating media i.e., Movies, TV-Series subtitle scraping from mult

Rafay 106 Dec 17, 2022
apkizer is a mass downloader for android applications for all available versions.

apkizer apkizer collects all available versions of an Android application from apkpure.com Purpose Sometimes mobile applications can be useful to dig

Kamil Onur Özkaleli 41 Dec 16, 2022
File Downloader

File Downloader Watches a file containing download links and runs a command to download them. The link file is in form of: # comment DOWNLOAD_LINK

Pouriya 1 Jan 08, 2022
𝐴 𝑡𝑒𝑙𝑒𝑔𝑟𝑎𝑚 𝑏𝑜𝑡 𝑡ℎ𝑎𝑡 𝑐𝑎𝑛 𝑑𝑜𝑤𝑛𝑙𝑜𝑎𝑑 𝑣𝑖𝑑𝑒𝑜 𝑎𝑛𝑑 𝑎𝑢𝑑𝑖𝑜 𝑓𝑟𝑜𝑚 𝑦𝑜𝑢𝑡𝑢𝑏𝑒 𝑎𝑛𝑑 𝑣𝑖𝑑𝑒𝑜 𝑤𝑒𝑏𝑠𝑖𝑡𝑒𝑠 𝑞𝑢𝑖𝑐𝑘𝑙𝑦

𝐴 𝑡𝑒𝑙𝑒𝑔𝑟𝑎𝑚 𝑏𝑜𝑡 𝑡ℎ𝑎𝑡 𝑐𝑎𝑛 𝑑𝑜𝑤𝑛𝑙𝑜𝑎𝑑 𝑣𝑖𝑑𝑒𝑜 𝑎𝑛𝑑 𝑎𝑢𝑑𝑖𝑜 𝑓𝑟𝑜𝑚 𝑦𝑜𝑢𝑡𝑢𝑏𝑒 𝑎𝑛𝑑 𝑣𝑖𝑑𝑒𝑜 𝑤𝑒𝑏𝑠𝑖𝑡𝑒𝑠 𝑞𝑢𝑖𝑐𝑘𝑙𝑦

SOCIAL MECHANIC 2 Aug 04, 2022
Stremio addon for fetching videos from your google drive.

stremio-gdrive Instructions: There are two ways to go about: Method 1 is hard and long but might give you better performance and you need to make your

72 Dec 31, 2022
👻🟡 Download all Snapchat video & photo memories from a data export.

Snapchat "Memories" Fetcher In compliance with the California Consumer Privacy Act of 2018 (“CCPA”), businesses which collect and store user data must

Todd Birchard 18 Dec 26, 2022
Download candlestick data fast & easy for analysis

crypto-candlesticks 📈 The goal behind this project is to facilitate downloading cryptocurrency candlestick data fast & simple. Currently only the Bit

Pedro Torres 31 Dec 11, 2022
Python software to download videos from Tiktok without rights

download-video-tiktok Python software to download videos from Tiktok without rights to install pip install requests Follow us telegram : https://t.me

muntazir halim 1 Oct 28, 2021
Neon: an add-on for making it easier to handle component interactions

Neon Neon is an add-on for Lightbulb making it easier to handle component interactions. Installation pip install git+https://github.com/neonjonn/light

Neon Jonn 9 Apr 29, 2022
Twitter Media Downloader (Telegram Bot)

Twitter Media Downloader (Telegram Bot)

Matin Baloochestani 8 Oct 27, 2022
A Celery application to collect data, download media and extract information from social media APIs

Project IBEX A Celery application to collect data, download media and extract information from social media APIs. Requirements You must have a Redis D

ibex 4 Dec 15, 2022
A program that can download animations from myself website

MYD A program that can download animations from myself website 一個可以用來下載Myself網站上動漫的程式 Quick Start [無GUI版本] 確定電腦內包含 ffmpeg 並設為環境變數 (Environment Variabl

Patrick_star 1 Nov 07, 2021
YouTube Video Search Engine For Python

YouTube-Video-Search-Engine Introduction With the increasing demand for electronic devices, it is hard for people to choose the best products from mul

1 Dec 21, 2021
命令行版本的HLS/DASH流下载器,支持标准AES-128-CBC解密

XstreamDL-CLI 基于python 3.7.4+的,命令行版本的,HLS/DASH流下载器,支持标准AES-128-CBC解密 使用 首先安装必要的库

xhlove 239 Dec 31, 2022
抖音批量下载助手

抖音批量下载助手

HuangSK 303 Jan 05, 2023
code for paper"3D reconstruction method based on a generative model in continuous latent space"

PyTorch implementation of 3D-VGT(3D-VAE-GAN-Transformer) This repository contains the source code for the paper "3D reconstruction method based on a g

Tong 5 Apr 25, 2022
Download YouTube videos/music and images in MP4, JPG with this tool.

ABOUT THE TOOL Download YouTube videos, music and images in MP4, JPG with this tool, with an easy to understand interface. This tool works with both,

TrollSkull 5 Jan 02, 2023