Fastest python library for making asynchronous group requests.

Overview

FGrequests: Fastest Asynchronous Group Requests

PyPI version fury.io PyPI pyversions License Donate

Installation

Install using pip:

pip install fgrequests

Documentation

Pretty easy to use.

import fgrequests

urls = [
    'https://google.com',
    'https://facebook.com',
    'https://twitter.com',
    'https://linkedin.com',
    'https://fakedomain.com'
]

Now lets make requests at the same time to the list of URLs (urls)

>>> response = fgreuests.build(urls)
>>> print(response)
[<Response [200]>, <Response [200]>, <Response [200]>, <Response [200]>, None]

By default fgrequests.build() returns a list of responses. If there have any invalid URL, the response will be None.

Method

By default this build() using GET method. There is a parameter which accepts methods named method. You can change this according to your need. method will accept these: GET, POST, PUT, DELETE, PATCH.

Lets send POST request in all of the urls

>>> response = fgreuests.build(urls, method='POST')
>>> print(response)
[<Response [405]>, <Response [200]>, <Response [200]>, <Response [200]>, None]

Headers

If you want to pass any headers you can simply pass your headers object (which may contain the authentication information) if you do like this:

>>> headers = {
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36',
    "Authorization": "Bearer XXXXXXXXXXXXXXXTOKEN"
}
>>> response = fgreuests.build(urls, headers=headers)

Params/Payload

If you want to pass additional information while making requests, just pass your params / payload by following way:

>>> data = {
    'username': 'farid'
    'password': 'password123'
}
>>> response = fgreuests.build(urls, data=data)

Proxies

If you need to use a proxy, you can configure individual requests with the proxies argument to any request method:

>>> proxies = {
    'http': 'http://10.10.1.10:3128',
    'https': 'http://10.10.1.10:1080',
}
>>> response = fgreuests.build(urls, proxies=proxies)

To use HTTP Basic Auth with your proxy, use the http://user:[email protected]/ syntax:

>>> proxies = {'http': 'http://user:[email protected]:3128/'}

To give a proxy for a specific scheme and host, use the scheme://hostname form for the key. This will match for any request to the given scheme and exact hostname.

>>> proxies = {'http://10.20.1.128': 'http://10.10.1.10:5323'}

SSL Cert Verification

Requests verifies SSL certificates for HTTPS requests, just like a web browser. By default, SSL verification is enabled, and Requests will throw a SSLError if it’s unable to verify the certificate. Requests can also ignore verifying the SSL certificate if you set verify to False:

>>> response = fgrequests.build(urls, verify=False)

For more info: SSL Cert Verification

Worker

It has another parameter worker. By default the value of worker is 40. If you increase this it will work more faster. But there is a problem if you increase this too much, this will make a lot of pressure in your CPU cores which may freeze your system. If you reduce the value of worker you this will take more time to return responses. You can change the value of worker like this:

>>> response = fgreuests.build(urls, worker=70)

POST a Multipart-Encoded File

You can simply to upload Multipart-encoded files by using files parameter:

>>> urls = ['https://httpbin.org/post']
>>> files = {'file': open('report.xls', 'rb')}

>>> response = fgrequests.build(urls, method='POST', files=files)
>>> response[0].text
{
  ...
  "files": {
    "file": "<censored...binary...data>"
  },
  ...
}

You can set the filename, content_type and headers explicitly:

>>> urls = ['https://httpbin.org/post']
>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}

>>> response = fgrequests.build(urls, method='POST', files=files)
>>> response[0].text
{
  ...
  "files": {
    "file": "<censored...binary...data>"
  },
  ...
}

If you want, you can send strings to be received as files:

>>> urls = ['https://httpbin.org/post']
>>> files = {'file': ('report.csv', 'some,data,to,send\nanother,row,to,send\n')}

>>> response = fgrequests.build(urls, method='POST', files=files)
>>> response[0].text
{
  ...
  "files": {
    "file": "some,data,to,send\\nanother,row,to,send\\n"
  },
  ...
}

Timeout

You can set timeout for the group request by using another parameter timeout. By default the value of timeout is 3 which is in seconds. You can change the value of timeout like this:

>>> response = fgreuests.build(urls, timeout=5)

Max Retries

You can put the count of maximum retries (to handle worst scenario) by using max_retries parameter. By default the value of max_retries is 1. You can change the value of max_retries like this:

>>> response = fgreuests.build(urls, max_retries=3)

Allow Redirects

You can disable redirection handling with the allow_redirects parameter. It only accepts Boolean, either True or False. By default it is True. You can change this by following way:

>>> response = fgreuests.build(urls, allow_redirects=False)

Execution Time

There have another parameter named show_execution_time. It returns the execution time (in sec). It accepts Boolean, either True or False. By default it is False. If you change this to True then fgrequests.build() will return an object. Lets check the output by making show_execution_time to True:

>>> response = fgrequests.build(urls, show_execution_time=True)
>>> print(response)
{
    'response_list': [<Response [200]>, <Response [200]>, <Response [200]>, <Response [200]>, None],
    'execution_time': 1.677
}

Support

You may report bugs, ask for help, and discuss various other issues on the bug tracker.

Donation

If this project help you reduce time to develop, you can give me a cup of coffee 🙂

paypal

You might also like...
Randomly distribute members by groups making sure that every sector is represented

Generate Groups Randomly distribute members by groups making sure that every sector is represented The Scenario Imagine that you have a large group of

New multi tool im making adding features currently

Emera Multi Tool New multi tool im making adding features currently Current List of Planned Features - Linkvertise Bypasser - Discord Auto Bump - Gith

A Lite Package focuses on making overwrite and mending functions easier and more flexible.

Overwrite Make Overwrite More flexible In Python A Lite Package focuses on making overwrite and mending functions easier and more flexible. Certain Me

an opensourced roblox group finder writen in python 100% free and virus-free

Roblox-Group-Finder an opensourced roblox group finder writen in python 100% free and virus-free note : if you don't want install python or just use w

TeamFleming is a multicultural group of 20 young bioinformatics enthusiasts participating in the 2021 HackBio Virtual Summer Internship
TeamFleming is a multicultural group of 20 young bioinformatics enthusiasts participating in the 2021 HackBio Virtual Summer Internship

💻 Welcome to Team Fleming's Repo! #TeamFleming is a multicultural group of 20 young bioinformatics enthusiasts participating in the 2021 HackBio Virt

Team Curie is a group of people working together to achieve a common aim
Team Curie is a group of people working together to achieve a common aim

Team Curie is a group of people working together to achieve a common aim. We are enthusiasts!.... We are setting the pace!.... We offer encouragement and motivation....And we believe TeamWork makes the DreamWork.

Group P-11's submission for the University of Waterloo's 2021 Engineering Competition (Programming section).
Group P-11's submission for the University of Waterloo's 2021 Engineering Competition (Programming section).

P-11-WEC2021 Group P-11's submission for the University of Waterloo's 2021 Engineering Competition (Programming section). Part I Compute typing time f

A discord group chat creator just made it because i saw people selling this stuff for like up to 40 bucks

gccreator some discord group chat tools just made it because i saw people selling this stuff for like up to 40 bucks (im currently working on a faster

Video Stream is an Advanced Telegram Bot that's allow you to play Video & Music on Telegram Group Video Chat
Video Stream is an Advanced Telegram Bot that's allow you to play Video & Music on Telegram Group Video Chat

Video Stream is an Advanced Telegram Bot that's allow you to play Video & Music on Telegram Group Video Chat 📊 Stats 🧪 Get SESSION_NAME from below:

Releases(0.1.0)
Owner
Farid Chowdhury
Python | Django
Farid Chowdhury
COVID-19 case tracker in Dash

covid_dashy_personal This is a personal project to build a simple COVID-19 tracker for Australia with Dash. Key functions of this dashy will be to Dis

Jansen Zhang 1 Nov 30, 2021
A carrot-based color palette you didn't know you needed.

A package to produce a carrot-inspired color palette for python/matplotlib. Install: pip install carrotColors Update: pip install --upgrade carrotColo

10 Sep 28, 2021
Forward RSS feeds to your email address, community maintained

Getting Started With rss2email We highly recommend that you watch the rss2email project on GitHub so you can keep up to date with the latest version,

248 Dec 28, 2022
Xbox-Flood is for flood anything

Intruduction Installation Usage Installing Python 3 Wiki Getting Started Creating a Key Intruduction Xbox-Flood is for flooding messages (invitations

kayake 4 Feb 18, 2022
This is the code of Python enthusiasts collection and written.

I am Python's enthusiast, like to collect Python's programs and code.

cnzb 35 Apr 18, 2022
This is a Docker-based pipeline for preparing sextractor-ready multiwavelength images

Pipeline for creating NB422-detected (ODI) catalog The repository contains a Docker-based pipeline for preprocessing observational data. The pipeline

1 Sep 01, 2022
Ingest openldap data into bloodhound

Bloodhound for Linux Ingest a dumped OpenLDAP ldif into neo4j to be visualized in Bloodhound. Usage: ./ldif_to_neo4j.py ./sample.ldif | cypher-shell -

Guillaume Quéré 71 Nov 09, 2022
My attempt at this years Advent of Code!

Advent-of-code-2021 My attempt at this years Advent of Code! day 1: ** day 2: ** day 3: ** day 4: ** day 5: ** day 6: ** day 7: ** day 8: * day 9: day

1 Jul 06, 2022
Beancount: Double-Entry Accounting from Text Files.

beancount: Double-Entry Accounting from Text Files Contents Description Documentation Download & Installation Versions Filing Bugs Copyright and Licen

2.3k Dec 28, 2022
Learning objective: Use React.js, Axios, and CSS to build a responsive YouTube clone app

Learning objective: Use React.js, Axios, and CSS to build a responsive YouTube clone app to search for YouTube videos, channels, playlists, and live events via wrapper around Google YouTube API.

Dillon 0 May 03, 2022
🤞 Website-Survival-Detection

- 🤞 Website-Survival-Detection It can help you to detect the survival status of the website in batches and return the status code! - 📜 Instructions

B0kd1 4 Nov 14, 2022
📜Generate poetry with gcc diagnostics

gado (gcc awesome diagnostics orchestrator) is a wrapper of gcc that outputs its errors and warnings in a more poetic format.

Dikson Santos 19 Jun 25, 2022
BlueBorne Dockerized

BlueBorne Dockerized This is the repo to reproduce the BlueBorne kill-chain on Dockerized Android as described here, to fully understand the code you

SecSI 5 Sep 14, 2022
Sigma coding youtube - This is a collection of all the code that can be found on my YouTube channel Sigma Coding.

Sigma Coding Tutorials & Resources YouTube • Facebook Support Sigma Coding Patreon • GitHub Sponsor • Shop Amazon Table of Contents Overview Topics Re

Alex Reed 927 Jan 08, 2023
importlib_resources is a backport of Python standard library importlib.resources module for older Pythons.

importlib_resources is a backport of Python standard library importlib.resources module for older Pythons. The key goal of this module is to replace p

Python 36 Dec 13, 2022
Choice Coin 633 Dec 23, 2022
Shared utility scripts for AI for Earth projects and team members

Overview Shared utilities developed by the Microsoft AI for Earth team The general convention in this repo is that users who want to consume these uti

Microsoft 38 Dec 30, 2022
Tools for dos (denial-of-service) website / web server

DoS Attack Tools Tools for dos (denial-of-service) website / web server di buat olah NurvySec How to install on debian / ubuntu $ apt update $ apt ins

nurvy 1 Feb 10, 2022
Tesla App Update Differences Extractor

Tesla App Update Differences Extractor Python program that finds the differences between two versions of the Tesla App. When Tesla updates the app a l

Adrian 5 Apr 11, 2022
A play store search module

A play store search module

Fayas Noushad 5 Dec 01, 2021