Python wrapper for Xeno-canto API 2.0. Enables downloading bird data with one command line

Overview

birdData

BirdData is a python wrapper for Xeno-canto API 2.0. Enables user to download bird data with one command line. BirdData supports multithreading download.

Environment

Download repo to local:

git clone [email protected]:realzza/birdData.git

Set up environment:

pip install -r requirement.txt

Usage

Single-thread

Download audio data for one bird species. Use scientific name starting with lowercase. e.g, cettia cetti.

python download.py --name "cettia cetti"

Download audio data for a file of species names. Format requirement: names divided by "\n"

python download.py --name name_file

General Usage:

usage: download.py [-h] --name NAME

download bird audios

optional arguments:
  -h, --help   show this help message and exit
  --name NAME  [1] name of one bird species; [2] file of bird species spaced
               by '\n'

Multi-thread

Speed up downloading using multiple threads.

python download-mult.py --name "cettia cetti" --process-ratio 0.6

Download multiple birds in a file, format requirement: names divided by "\n"

python download-mult.py --name name_file --process-ratio 0.6

General Usage:

usage: download-mult.py [-h] --name NAME [--process-ratio PROCESS_RATIO]

download bird audios

optional arguments:
  -h, --help            show this help message and exit
  --name NAME           [1] name of one bird species; [2] file of bird species
                        spaced by '\n'
  --process-ratio PROCESS_RATIO
                        float[0~1], define cpu utilities in downloading audios
                        [default: 0.8]

To-do

  • [12.29] multiprocess download
  • define sample rate prior to download

Contact

Feel free to file an issue had you encountered any problems. Have fun!

You might also like...
PRAW, an acronym for "Python Reddit API Wrapper", is a python package that allows for simple access to Reddit's API.

PRAW: The Python Reddit API Wrapper PRAW, an acronym for "Python Reddit API Wrapper", is a Python package that allows for simple access to Reddit's AP

A Python Instagram Scraper for Downloading Profile's Posts, stories, ProfilePic and See the Details of Particular Instagram Profile.
A Python Instagram Scraper for Downloading Profile's Posts, stories, ProfilePic and See the Details of Particular Instagram Profile.

✔ ✔ InstAstra ⚡ ⚡ ⁜ Description ~ A Python Instagram Scraper for Downloading Profile's Posts, stories, ProfilePic and See the Details of Particular In

Python API wrapper around Trello's API

A wrapper around the Trello API written in Python. Each Trello object is represented by a corresponding Python object. The attributes of these objects

Async ready API wrapper for Revolt API written in Python.

Mutiny Async ready API wrapper for Revolt API written in Python. Installation Python 3.9 or higher is required To install the library, you can just ru

A Python API wrapper for the Twitter API!

PyTweet PyTweet is an api wrapper made for twitter using twitter's api version 2! Installation Windows py3 -m pip install PyTweet Linux python -m pip

Python API wrapper library for Convex Value API

convex-value-python Python API wrapper library for Convex Value API. Further Links: Convex Value homepage @ConvexValue on Twitter JB on Twitter Authen

This an API wrapper library for the OpenSea API written in Python 3.

OpenSea NFT API Python 3 wrapper This an API wrapper library for the OpenSea API written in Python 3. The library provides a simplified interface to f

YARSAW is an Async Python API Wrapper for the Random Stuff API.

Yet Another Random Stuff API Wrapper - YARSAW YARSAW is an Async Python API Wrapper for the Random Stuff API. This module makes it simpler for you to

EpikCord.py - This is an API Wrapper for Discord's API for Python

EpikCord.py - This is an API Wrapper for Discord's API for Python! We've decided not to fork discord.py and start completely from scratch for a new, better structuring system!

Comments
  • ZeroDivisionError

    ZeroDivisionError

    您好,我在尝试批量下载的时候,发现当查询结果为0个的时候,会出现 ZeroDivisionError: integer division or modulo by zero 的错误,查询trackback,错误源头在utils.py文件的第4行portion = len(lst) //n的位置。

    除此之外,我还发现当查询以下物种名称的时候也会出现同样的error。我的查询时间范围是从1971-01-01:

    ['Maleo', 'Moluccan Megapode', 'Nicobar Megapode', 'Long-billed Partridge', 'Black Partridge', 'Udzungwa Forest Partridge', 'Rubeho Forest Partridge', "Roll's Partridge", 'Sumatran Partridge', 'Grey-breasted Partridge', 'Red-billed Partridge', 'Chestnut-necklaced Partridge', 'Crestless Fireback', 'Crested Fireback', 'Siamese Fireback', 'Great Argus', 'Madagascan Pochard']
    

    以下这个list是对应以上的common_name的查询结果数量

    [7, 6, 2, 18, 3, 6, 34, 10, 1, 31, 8, 35, 1, 17, 24, 83, 5]
    

    期待回复,感谢!

    small fix 
    opened by ZhongJunhong 1
  • Something about Download Timeout

    Something about Download Timeout

    您好,我在下载的时候发现一些音频文件在某些情况下(可能是网络的问题)下载时间过长,而且进度条一直停滞不前,有时甚至停滞十分钟到二十分钟不等,有时候我只能手动中断使之进入下一个循环。

    后来我在stackoverflow上找到一个非常非常简单的解决方案,就是在执行请求的代码前加入socket.setdefaulttimeout(30),控制socket打开的时间,比如此处我设置为30秒。

    socket.setdefaulttimeout(30) 
    q.retrieve_recordings(multiprocess=True, nproc=10, attempts=10, outdir="/mnt/database/xcdata/")
    

    同时我加入一些其他设置,让代码在触发socket.timeout错误后,可以将对应的请求参数记录下来,并继续执行下一个循环。由此可以避免下载停滞的问题。当我完成所有循环以后,会将socket打开的时间再增加(比如增加到120秒),对先前记录下的请求参数再次执行。由此循环多次以将所有参数请求完毕。

    https://stackoverflow.com/questions/32763720/timeout-a-file-download-with-python-urllib

    希望能帮助到大家。

    Hello, when I was downloading, I found that the download time of some audio files was too long under certain circumstances (may be a problem with the network), and the progress bar has been stagnant, sometimes even for ten to twenty minutes, there are Sometimes I can only manually interrupt it to enter the next cycle.

    Later, I found a very, very simple solution on stackoverflow, which is to add socket.setdefaulttimeout(30) before executing the requested code to control the opening time of the socket. For example, I set it to 30 seconds here.

    socket.setdefaulttimeout(30) 
    q.retrieve_recordings(multiprocess=True, nproc=10, attempts=10, outdir="/mnt/database/xcdata/")
    

    At the same time, I added some other settings so that after the code triggers the socket.timeout error, it can record the corresponding request parameters and continue to execute the next loop. This can avoid the problem of download stagnation. When I finish all the loops, I will increase the socket opening time (for example, to 120 seconds), and execute again on the previously recorded request parameters. Loop for multiple times to complete the request for all parameters.

    https://stackoverflow.com/questions/32763720/timeout-a-file-download-with-python-urllib

    I Hope that can help everyone.

    opened by ZhongJunhong 3
Releases(v0.0.4)
  • v0.0.4(May 13, 2022)

    • Support Query by bird name.
    • Automatically cut inessential processes in query traffic.
    • Optimized query assignment strategy in recording retrieval.
    Source code(tar.gz)
    Source code(zip)
Owner
_zza
Computational (Para)linguistics | Multimodal Analysis | Founder of @Presento | Previously @ByteDance AI-lab Shanghai
_zza
This is a free python bot program that crosses you to farm with auto click in space crypto NFT game, having fun :) Creator: Marlon Zanardi

🚀 Space Crypto auto click bot ready-to-use 🚀 This is a free python bot program that crosses you to farm with auto click in space crypto NFT game, ha

170 Dec 20, 2022
An advanced crypto trading bot written in Python

Jesse Jesse is an advanced crypto trading framework which aims to simplify researching and defining trading strategies. Why Jesse? In short, Jesse is

Jesse 4.4k Jan 09, 2023
Project developed as part of a selection process for the company Denox

📝 Tabela de conteúdos Sobre Requisitos para rodar o projeto Instalação Rotas da API Observações 🧐 Sobre Projeto desenvolvido como parte de um proces

Ícaro Sant'Ana 1 Mar 01, 2022
🚀🔥使用Python连接阿里云盘, 实现了官方大部分功能 👍👍

aligo 🚀 🔥 使用Python连接阿里云盘, 实现了官方大部分功能 👍 👍 为了完善代码提示, 方便大家代码书写, aligo 引入了一些 python 3.8 的新特性, 所以要求 python = 3.8.* pip install aligo 或 pip install ali

455 Jan 08, 2023
A Python wrapper for the Yelp API v2

python-yelp-v2 A Python wrapper for the Yelp API v2. The structure for this was inspired by the python-twitter library, and some internal methods are

Matthew Conlen 12 Oct 24, 2017
Crosschat - A bot for cross-server communication

CrossChat A bot for cross-server communication. Running the bot To run the bot y

8 May 15, 2022
Pluggable Telethon - Telegram UserBot

A stable pluggable Telegram userbot, based on Telethon.

Team Ultroid 2.3k Dec 30, 2022
Un petit tool qui est la pour envoier des message avec des webhook en bêta

📎 Webhook-discord Le but de se tool c'est que tu peux envoier vos webhook discord sur vos serveur et les customiser Pour lancer le projet il faut avo

2 Oct 10, 2021
Telegram bot that sends new offers from otomoto.pl

Telegram bot that sends new offers under certain filters from otomoto.pl How to use this bot? Install requirements with pip install -r requirements.tx

Mikhail Zanka 1 Feb 14, 2022
Andrei 1.4k Dec 24, 2022
Discord-Token-Formatter - A simple script to convert discord tokens from email token to token only format

Discord-Token-Formatter A simple script to convert discord tokens from email:pas

2 Oct 23, 2022
A Twitch bot to provide information from the WebNowPlayingCompanion extension

WebNowPlayingTwitch A Twitch bot made using TwitchIO which displays information obtained from the WebNowPlaying Compaion browser extension. Image is o

NiceAesth 1 Mar 21, 2022
A module to get data about anime characters, news, info, lyrics and more.

Animec A module to get data about anime characters, news, info, lyrics and more. The module scrapes myanimelist to parse requested data. If you wish t

DriftAsimov 31 Aug 31, 2022
Tools ini hanya bisa digunakan untuk menyerang website atau http/s

☢️ Tawkun DoS ☢️ Tools ini hanya bisa digunakan untuk menyerang website atau http/s FITUR: [ ☯️ ] Proxy Mode [ 🔥 ] SOCKS Mode | Kadang Eror [ ☢️ ] Ht

Bandhitawkunthi 9 Jul 19, 2022
A Discord bot to combat phishing links for Steam trades and Discord gifts.

delink-bot A Discord bot to combat phishing links for Steam trades and Discord gifts. Requirement python3 -m pip install -U discord.py python3 -m pip

hugonun 15 Dec 09, 2022
This repository contains modules that extend / modify parts of Odoo ERP

Odoo Custom Addons This repository contains addons that extend / modify parts of Odoo ERP. Addons list account_cancel_permission Only shows the button

Daniel Luque 3 Dec 28, 2022
A feishu bot daily push arxiv latest articles.

arxiv-feishu-bot We develop A simple feishu bot script daily pushes arxiv latest articles. His effect is as follows: Of course, you can also use other

huchi 6 Apr 06, 2022
Get random jokes bapack2 on telegram

Jokes Bapack2 Telegram Bot Get random jokes bapack2 from jokes-bapack2-api on telegram bot Screenshot Requirements python pip pipenv python-telegram-b

Miftah Afina 2 Nov 17, 2021
A cracking tool of Xiaomi Dr AI (Archytas / Archimedes)

Archytas Tool 我们强烈抵制闲鱼平台上未经授权的刷机服务! 我对本人之前在程序中为防止违规刷机服务添加未生效的格机代码感到抱歉,在此声明此过激行为与 Crack Mi Dr AI Team 无关,并将程序开源。 A cracking tool of Xiaomi Dr AI (Archy

rponeawa 5 Oct 25, 2022
2b2t Priority queue discord bot announcer

2b2t Priority queue discord bot announcer Commands !prioq - Checks the priority queue length and sends it. !start - Starts a loop that sends the sta

Gumi 5 Jun 06, 2022