The official wrapper for spyse.com API, written in Python, aimed to help developers build their integrations with Spyse.

Overview

Python wrapper for Spyse API

The official wrapper for spyse.com API, written in Python, aimed to help developers build their integrations with Spyse.

Spyse is the most complete Internet assets search engine for every cybersecurity professional.

Examples of data Spyse delivers:

  • List of 300+ most popular open ports found on 3.5 Billion publicly accessible IPv4 hosts.
  • Technologies used on 300+ most popular open ports and IP addresses and domains using a particular technology.
  • Security score for each IP host and website, calculated based on the found vulnerabilities.
  • List of websites hosted on each IPv4 host.
  • DNS and WHOIS records of the domain names.
  • SSL certificates provided by the website hosts.
  • Structured content of the website homepages.
  • Abuse reports associated with IPv4 hosts.
  • Organizations and industries associated with the domain names.
  • Email addresses found during the Internet scanning, associated with a domain name.

More information about the data Spyse collects is available on the Our data page.

Spyse provides an API accessible via token-based authentication. API tokens are available only for registered users on their account page.

For more information about the API, please check the API Reference.

Installation

pip3 install spyse-python

Updating

pip3 install --no-cache-dir spyse-python

Quick start

from spyse import Client

client = Client("your-api-token-here")

d = client.get_domain_details('tesla.com')

print(f"Domain details:")
print(f"Website title: {d.http_extract.title}")
print(f"Alexa rank: {d.alexa.rank}")
print(f"Certificate subject org: {d.cert_summary.subject.organization}")
print(f"Certificate issuer org: {d.cert_summary.issuer.organization}")
print(f"Updated at: {d.updated_at}")
print(f"DNS Records: {d.dns_records}")
print(f"Technologies: {d.technologies}")
print(f"Vulnerabilities: {d.cve_list}")
print(f"Trackers: {d.trackers}")
# ...

Examples

Note: You need to export access_token to run any example:

export SPYSE_API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

How to search

Using Spyse you can search for any Internet assets by their digital fingerprints. To do that, you need to form a specific search query and pass it to 'search', 'scroll', or 'count' methods.

Each search query can contain multiple search params. Each search param consists of name, operator, and value.

Check API docs to find out all existing combinations. Here is an example for domains search: https://spyse-dev.readme.io/reference/domains#domain_search You may also be interested in our GUI for building and testing queries before jumping to code: https://spyse.com/advanced-search/domain

Example search request to find subdomains of att.com:

from spyse import Client, SearchQuery, QueryParam, DomainSearchParams, Operators

# Prepare query
q = SearchQuery()
domain = "att.com"

# Add param to search for att.com subdomains
q.append_param(QueryParam(DomainSearchParams.name, Operators.ends_with, '.' + domain))

# Add param to search only for alive subdomains
q.append_param(QueryParam(DomainSearchParams.http_extract_status_code, Operators.equals, "200"))

# Add param to remove subdomains seen as PTR records
q.append_param(QueryParam(DomainSearchParams.is_ptr, Operators.equals, "False"))

# Next, you can use the query to run search, count or scroll methods
c = Client("your-api-token-here")
total_count = c.count_domains(q)
search_results = c.search_domains(q).results
scroll_results = c.scroll_domains(q).results

Example search request to find any alive IPv4 hosts in US, with open port 22 and running nginx:

from spyse import Client, SearchQuery, QueryParam, IPSearchParams, Operators

# Prepare query
q = SearchQuery()

# Add param to search for IPv4 hosts located in US
q.append_param(QueryParam(IPSearchParams.geo_country_iso_code, Operators.equals, 'US'))

# Add param to search only for hosts with open 22 port
q.append_param(QueryParam(IPSearchParams.open_port, Operators.equals, "22"))

# Add param to search only for hosts with nginx
q.append_param(QueryParam(IPSearchParams.port_technology_name, Operators.contains, "nginx"))

# Next, you can use the query to run search, count or scroll methods
c = Client("your-api-token-here")
total_count = c.count_ip(q)
search_results = c.search_ip(q).results
scroll_results = c.scroll_ip(q).results

Scroll vs Search

While a 'search' request allows to paginate over the first 10'000 results, the 'scroll search' can be used for deep pagination over a larger number of results (or even all results) in much the same way as you would use a cursor on a traditional database.

In order to use scrolling, the initial search response will return a 'search_id' data field which should be specified in the subsequent requests in order to iterate over the rest of results.

Limitations

The scroll is available only for customers with 'Pro' subscription.

Example code to check if the scroll is available for your account

from spyse import Client
c = Client("your-api-token-here")

if c.get_quotas().is_scroll_search_enabled:
    print("Scroll is available")
else:
    print("Scroll is NOT available")

Development

Installation

git clone https://github.com/spyse-com/spyse-python
pip install -e .

Run tests:

cd tests
python client_test.py

License

Distributed under the MIT License. See LICENSE for more information.

Troubleshooting and contacts

For any proposals and questions, please write at:

Comments
  • Update deps and add a dependabot.yml file

    Update deps and add a dependabot.yml file

    This PR is to fix issues with spyse using out of date deps and allow projects that use this SDK to be able to install new versions of deps that they use by fixing sypse SDK to use the latest of its deps. If this PR gets accepted can you then please tag a new release to pypi with these changes, thanks

    opened by L1ghtn1ng 0
  • Domain Lookup doesn't return result

    Domain Lookup doesn't return result

    As the issue mentioned, i'm trying to experiment with spyse-py to automate my search queries. Using the script template (Already input the token-key and some domain name example) it wont show the result. Screenshot_2022-04-30-15-09-35-27

    opened by MC189 0
  • Update license entry

    Update license entry

    It should be the license identifier and not the license text.

    At the moment PyPI shows the content of the LICENSE.md file.

    This would also allow third-party tools (e. g., pyp2rpm) to get the license details in a simple way.

    opened by fabaff 0
  • spyse.response.RateLimitError: too many requests

    spyse.response.RateLimitError: too many requests

    from spyse import Client
    
    
    def main():
        client = Client("token")
        q = client.get_quotas()
    
    
    if __name__ == '__main__':
        main()
    
    

    Traceback:

    Traceback (most recent call last):
      File "/home/name/py-trash/privatbank/main.py", line 11, in <module>
        main()
      File "/home/name/py-trash/privatbank/main.py", line 7, in main
        q = client.get_quotas()
      File "/home/name/py-trash/venv/lib/python3.9/site-packages/spyse/client.py", line 106, in get_quotas
        response.check_errors()
      File "/home/name/py-trash/venv/lib/python3.9/site-packages/spyse/response.py", line 117, in check_errors
        raise RateLimitError(m)
    spyse.response.RateLimitError: too many requests
    
    

    Python 3.9.9

    Package            Version
    ------------------ ---------
    certifi            2021.10.8
    charset-normalizer 2.0.11
    dataclasses        0.6
    dataclasses-json   0.5.6
    idna               3.3
    limiter            0.1.2
    marshmallow        3.14.1
    marshmallow-enum   1.5.1
    mypy-extensions    0.4.3
    pip                21.2.4
    requests           2.26.0
    responses          0.13.4
    setuptools         58.0.4
    six                1.16.0
    spyse-python       2.2.3
    token-bucket       0.2.0
    typing_extensions  4.0.1
    typing-inspect     0.7.1
    urllib3            1.26.8
    wheel              0.37.0
    
    opened by fabelx 2
  • make dataclasses optional

    make dataclasses optional

    Hi, can you make dataclasses conditional, since it is a backport for Python 3.6 and included in newer Python versions?

    https://github.com/spyse-com/spyse-python/blob/main/requirements.txt#L2 https://github.com/spyse-com/spyse-python/blob/main/setup.py#L21

    Thanks

    opened by blshkv 0
Releases(v2.2.3)
Owner
Spyse
Internet assets search engine
Spyse
python3.5+ hubspot client based on hapipy, but modified to use the newer endpoints and non-legacy python

A python wrapper around HubSpot's APIs, for python 3.5+. Built initially around hapipy, but heavily modified. Check out the documentation here! (thank

Jacobi Petrucciani 140 Dec 21, 2022
An automated bot for twitter using Tweepy!

Tweeby An automated bot for twitter using Tweepy! About This bot will look for tweets that contain certain hashtags, if found. It'll send them a messa

Ori 1 Dec 06, 2021
discord.xp Bot, counts XP for members

discord.xp Bot, counts XP for members. How to setup and run? You must have an mysql database Download libs from the requirements.txt file Configurize

irwing 4 Feb 05, 2022
A Python Discord bot project generator

Heater Heat up a Discord bot in a blink What is Heater? Heater is a Command Line Interface tool which allows you to generate a barebones Python Discor

DevGuyAhnaf 5 Jan 14, 2022
A WhatsApp Crashing Tool for Termux

CrashW A WhatsApp Crashing Tool For Termux Users Installing : apt update && apt upgrade -y pkg install python3 pkg install git git clone git://gith

Gokul Mahato 20 Dec 27, 2022
GUI Pancakeswap V2 and Uniswap V3 trading client (and bot) MOST ADVANCE TRADING BOT SUPPORT WINDOWS LINUX MAC (BUY TOKEN ON LAUNCH)

GUI Pancakeswap 2 and Uniswap 3 SNIPER BOT 🏆 🥇 (MOST ADVANCE TRADING BOT SUPPORT WINDOWS LINUX MAC) (AUTO BUY TOKEN ON LAUNCH AFTER ADD LIQUIDITY) S

HYDRA 16 Dec 22, 2021
Easily update resume to naukri with one click

NAUKRI RESUME AUTO UPDATER I am using poetry for dependencies. you can check or change in data.txt file for username and password Resume file must be

Rahul.p 1 May 02, 2022
A python library to interact with the EarnApp API

EarnApp.py Table of contents General info Documentation Setup General info A python library to interact with the EarnApp API. Documentation First, imp

3 Dec 14, 2022
Python library for Seeedstudio Grove devices

grove.py Python library for Seeedstudio Grove Devices on embeded Linux platform, especially good on below platforms: Coral Dev Board (Wiki) NVIDIA Jet

Seeed Studio 123 Dec 17, 2022
A robust, low-level connector for the Discord API

Bauxite Bauxite is a robust, low-level connector for the Discord API. What is Bauxite for? Bauxite is made for two main purposes: Creating higher-leve

1 Dec 04, 2021
Discord Streaming Statut (Bot/SelfBot)

Discord-Streaming-Status Discord Streaming Status For Both User Accounts And Bot Accounts. Open your cmd and enter the command: pip install discord BE

Komuro 2 Nov 02, 2021
A program used to create accounts in bulk, still a work in progress as of now.

Discord Account Creator This project is still a work in progress. It will be published upon its full completion. About This project is still under dev

patched 8 Sep 15, 2022
Repositorio dedicado a contener los archivos fuentes del bot de discord "Lector de Ejercicios".

Lector de Ejercicios Este bot de discord está pensado para usarse únicamente en el discord de la materia Algoritmos y Programación I, de la Facultad d

Franco Lighterman Reismann 3 Sep 17, 2022
Send Informative, Concise Slack Notifications With Minimal Effort

slack-templates Send Informative, Concise Slack Notifications With Minimal Effort slack-templates Slack Integration Available Templates Usage Report t

9 Nov 03, 2022
A Python module for communicating with the Twilio API and generating TwiML.

twilio-python The default branch name for this repository has been changed to main as of 07/27/2020. Documentation The documentation for the Twilio AP

Twilio 1.6k Jan 05, 2023
Clash of Clans v6.253 private server written in python

cocps Clash of Clans v6.253 private server written in python how2play download server files download Patched APK run Main.py and play Authors Patched

5 Aug 28, 2022
nuub-bot is a multi-purpose designed and developed in python3

nuub-bot About Nuub-Bot is an open source, fully customizable Discord bot that is constantly growing. You can invite it to your Discord server using t

Baneet Parmar 8 Jun 28, 2022
An unofficial client library for Google Music.

gmusicapi: an unofficial API for Google Play Music gmusicapi allows control of Google Music with Python. from gmusicapi import Mobileclient api = Mob

Simon Weber 2.5k Dec 15, 2022
A reddit.com bot that will return reference links from official python documentation site for the standard library.

Python Docs Bot A reddit.com bot that will return documentation links for the library and language reference sections of the python docs website. The

Trevor Miller 2 Sep 14, 2021
Bancos de Dados Relacionais (SQL) na AWS com Amazon RDS

Bancos de Dados Relacionais (SQL) na AWS com Amazon RDS Repositório para o Live Coding DIO do dia 24/11/2021 Serviços utilizados Amazon RDS AWS Lambda

Cassiano Ricardo de Oliveira Peres 4 Jul 30, 2022