The official wrapper for spyse.com API, written in Python, aimed to help developers build their integrations with Spyse.

Overview

Python wrapper for Spyse API

The official wrapper for spyse.com API, written in Python, aimed to help developers build their integrations with Spyse.

Spyse is the most complete Internet assets search engine for every cybersecurity professional.

Examples of data Spyse delivers:

  • List of 300+ most popular open ports found on 3.5 Billion publicly accessible IPv4 hosts.
  • Technologies used on 300+ most popular open ports and IP addresses and domains using a particular technology.
  • Security score for each IP host and website, calculated based on the found vulnerabilities.
  • List of websites hosted on each IPv4 host.
  • DNS and WHOIS records of the domain names.
  • SSL certificates provided by the website hosts.
  • Structured content of the website homepages.
  • Abuse reports associated with IPv4 hosts.
  • Organizations and industries associated with the domain names.
  • Email addresses found during the Internet scanning, associated with a domain name.

More information about the data Spyse collects is available on the Our data page.

Spyse provides an API accessible via token-based authentication. API tokens are available only for registered users on their account page.

For more information about the API, please check the API Reference.

Installation

pip3 install spyse-python

Updating

pip3 install --no-cache-dir spyse-python

Quick start

from spyse import Client

client = Client("your-api-token-here")

d = client.get_domain_details('tesla.com')

print(f"Domain details:")
print(f"Website title: {d.http_extract.title}")
print(f"Alexa rank: {d.alexa.rank}")
print(f"Certificate subject org: {d.cert_summary.subject.organization}")
print(f"Certificate issuer org: {d.cert_summary.issuer.organization}")
print(f"Updated at: {d.updated_at}")
print(f"DNS Records: {d.dns_records}")
print(f"Technologies: {d.technologies}")
print(f"Vulnerabilities: {d.cve_list}")
print(f"Trackers: {d.trackers}")
# ...

Examples

Note: You need to export access_token to run any example:

export SPYSE_API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

How to search

Using Spyse you can search for any Internet assets by their digital fingerprints. To do that, you need to form a specific search query and pass it to 'search', 'scroll', or 'count' methods.

Each search query can contain multiple search params. Each search param consists of name, operator, and value.

Check API docs to find out all existing combinations. Here is an example for domains search: https://spyse-dev.readme.io/reference/domains#domain_search You may also be interested in our GUI for building and testing queries before jumping to code: https://spyse.com/advanced-search/domain

Example search request to find subdomains of att.com:

from spyse import Client, SearchQuery, QueryParam, DomainSearchParams, Operators

# Prepare query
q = SearchQuery()
domain = "att.com"

# Add param to search for att.com subdomains
q.append_param(QueryParam(DomainSearchParams.name, Operators.ends_with, '.' + domain))

# Add param to search only for alive subdomains
q.append_param(QueryParam(DomainSearchParams.http_extract_status_code, Operators.equals, "200"))

# Add param to remove subdomains seen as PTR records
q.append_param(QueryParam(DomainSearchParams.is_ptr, Operators.equals, "False"))

# Next, you can use the query to run search, count or scroll methods
c = Client("your-api-token-here")
total_count = c.count_domains(q)
search_results = c.search_domains(q).results
scroll_results = c.scroll_domains(q).results

Example search request to find any alive IPv4 hosts in US, with open port 22 and running nginx:

from spyse import Client, SearchQuery, QueryParam, IPSearchParams, Operators

# Prepare query
q = SearchQuery()

# Add param to search for IPv4 hosts located in US
q.append_param(QueryParam(IPSearchParams.geo_country_iso_code, Operators.equals, 'US'))

# Add param to search only for hosts with open 22 port
q.append_param(QueryParam(IPSearchParams.open_port, Operators.equals, "22"))

# Add param to search only for hosts with nginx
q.append_param(QueryParam(IPSearchParams.port_technology_name, Operators.contains, "nginx"))

# Next, you can use the query to run search, count or scroll methods
c = Client("your-api-token-here")
total_count = c.count_ip(q)
search_results = c.search_ip(q).results
scroll_results = c.scroll_ip(q).results

Scroll vs Search

While a 'search' request allows to paginate over the first 10'000 results, the 'scroll search' can be used for deep pagination over a larger number of results (or even all results) in much the same way as you would use a cursor on a traditional database.

In order to use scrolling, the initial search response will return a 'search_id' data field which should be specified in the subsequent requests in order to iterate over the rest of results.

Limitations

The scroll is available only for customers with 'Pro' subscription.

Example code to check if the scroll is available for your account

from spyse import Client
c = Client("your-api-token-here")

if c.get_quotas().is_scroll_search_enabled:
    print("Scroll is available")
else:
    print("Scroll is NOT available")

Development

Installation

git clone https://github.com/spyse-com/spyse-python
pip install -e .

Run tests:

cd tests
python client_test.py

License

Distributed under the MIT License. See LICENSE for more information.

Troubleshooting and contacts

For any proposals and questions, please write at:

Comments
  • Update deps and add a dependabot.yml file

    Update deps and add a dependabot.yml file

    This PR is to fix issues with spyse using out of date deps and allow projects that use this SDK to be able to install new versions of deps that they use by fixing sypse SDK to use the latest of its deps. If this PR gets accepted can you then please tag a new release to pypi with these changes, thanks

    opened by L1ghtn1ng 0
  • Domain Lookup doesn't return result

    Domain Lookup doesn't return result

    As the issue mentioned, i'm trying to experiment with spyse-py to automate my search queries. Using the script template (Already input the token-key and some domain name example) it wont show the result. Screenshot_2022-04-30-15-09-35-27

    opened by MC189 0
  • Update license entry

    Update license entry

    It should be the license identifier and not the license text.

    At the moment PyPI shows the content of the LICENSE.md file.

    This would also allow third-party tools (e. g., pyp2rpm) to get the license details in a simple way.

    opened by fabaff 0
  • spyse.response.RateLimitError: too many requests

    spyse.response.RateLimitError: too many requests

    from spyse import Client
    
    
    def main():
        client = Client("token")
        q = client.get_quotas()
    
    
    if __name__ == '__main__':
        main()
    
    

    Traceback:

    Traceback (most recent call last):
      File "/home/name/py-trash/privatbank/main.py", line 11, in <module>
        main()
      File "/home/name/py-trash/privatbank/main.py", line 7, in main
        q = client.get_quotas()
      File "/home/name/py-trash/venv/lib/python3.9/site-packages/spyse/client.py", line 106, in get_quotas
        response.check_errors()
      File "/home/name/py-trash/venv/lib/python3.9/site-packages/spyse/response.py", line 117, in check_errors
        raise RateLimitError(m)
    spyse.response.RateLimitError: too many requests
    
    

    Python 3.9.9

    Package            Version
    ------------------ ---------
    certifi            2021.10.8
    charset-normalizer 2.0.11
    dataclasses        0.6
    dataclasses-json   0.5.6
    idna               3.3
    limiter            0.1.2
    marshmallow        3.14.1
    marshmallow-enum   1.5.1
    mypy-extensions    0.4.3
    pip                21.2.4
    requests           2.26.0
    responses          0.13.4
    setuptools         58.0.4
    six                1.16.0
    spyse-python       2.2.3
    token-bucket       0.2.0
    typing_extensions  4.0.1
    typing-inspect     0.7.1
    urllib3            1.26.8
    wheel              0.37.0
    
    opened by fabelx 2
  • make dataclasses optional

    make dataclasses optional

    Hi, can you make dataclasses conditional, since it is a backport for Python 3.6 and included in newer Python versions?

    https://github.com/spyse-com/spyse-python/blob/main/requirements.txt#L2 https://github.com/spyse-com/spyse-python/blob/main/setup.py#L21

    Thanks

    opened by blshkv 0
Releases(v2.2.3)
Owner
Spyse
Internet assets search engine
Spyse
Flask extension that provides integration with Azure Storage

Flask-Azure-Storage A Flask extension that provides integration with Azure Storage Table of Contents Flask-Azure-Storage Install Usage Examples Create

Alejo Arias 17 Nov 14, 2021
A Twitter bot written in Python using Tweepy and hosted on a server.

A Twitter bot written in Python using Tweepy. It can like and/or retweet tweets that contain single or multiple keywords and hashtags.

anniedotexe 11 Dec 15, 2022
Jira-cache - Jira cache with python

Direct queries to Jira have two issues: they are sloooooow many queries are impo

John Scott 6 Oct 08, 2022
A hilarious program that translates text to Google's

Bork, bork, bork! A hilarious program that translates text in a file to Google's "bork bork bork' language. How to use Download the main.py file. Chan

The Algorithmic 1 Dec 17, 2021
A python script to download twitter space, only works on running spaces (for now).

A python script to download twitter space, only works on running spaces (for now).

279 Jan 02, 2023
Financial portfolio optimisation in python, including classical efficient frontier, Black-Litterman, Hierarchical Risk Parity

PyPortfolioOpt has recently been published in the Journal of Open Source Software 🎉 PyPortfolioOpt is a library that implements portfolio optimizatio

Robert Martin 3.2k Jan 02, 2023
Utilizing the freqtrade high-frequency cryptocurrency trading framework to build and optimize trading strategies. The bot runs nonstop on a Rasberry Pi.

Freqtrade Strategy Repository Please test all scripts and dry run them before using them in live mode Contact me on discord if you have any questions!

Michael Fourie 90 Jan 01, 2023
Plays air warning sound when detects a certain phrase or a word in a specified Telegram chat.

Tryvoha Bot Disclaimer: this is more a convenient naming, rather than a real bot. It is designed to play air warning sound when detects a certain phra

Dmytro Novikov 2 Mar 02, 2022
Super Fast Telegram UserBot Made With Python.

Description Super Fast Telegram UserBot Made With Python. LOGO Made With Support of All Userbots Dev's Dark-Venom is a Light-Weight Userbot. It's unde

2 Sep 14, 2021
Spacecrypto-bombcrypto-bot - SpaceCrypto And Bombcrypto Bot - MultiScreen

SpaceCrypto And Bombcrypto Bot - MultiScreen This is a open source project inspi

Paulo Bramante 5 Nov 03, 2022
A Telegram Userbot to play or streaming Audio and Video songs / files in Telegram Voice Chats.

Vcmusic-Userbot A Telegram Userbot to play or streaming Audio and Video songs / files in Telegram Voice Chats. It's made with PyTgCalls and Pyrogram R

3 Oct 23, 2021
A Telegram bot to extracting text from images. All languages supported.

OCR Bot A Telegram bot to extracting text from images. All languages supported. Deploy to Heroku Local Deploying Clone the repo git clone https://gith

6 Oct 21, 2022
A powerful application to automatically deploy GitHub Release.

A powerful application to automatically deploy GitHub Release.

Fentaniao 43 Sep 17, 2022
Telegram Bot For Screenshot Generation.

Screenshotit_bot Telegram Bot For Screenshot Generation. Description An attempt to implement the screenshot generation of telegram files without downl

1 Nov 06, 2021
Automated JSON API based communication with Fronius Symo

PyFronius - a very basic Fronius python bridge A package that connects to a Fronius device in the local network and provides data that is provided via

Niels Mündler 10 Dec 30, 2022
New discord token grabber, password and general information

New discord token grabber, password and general information

Monstered 6 Nov 09, 2022
100d002 - Simple program to calculate the tip amount and split the bill between all guests

Day 2 - Tip Calculator Simple program to calculate the tip amount and split the

Andre Schickhoff 1 Jan 24, 2022
This package accesses nitrotype's official api along with its unofficial user api

NitrotypePy This package accesses nitrotype's official api along with its unofficial user api. Currently still in development. Install To install, run

The Moon That Rises 2 Sep 04, 2022
An API wrapper around the pythonanywhere's API.

pyaww An API wrapper around the pythonanywhere's API. The name stands for pythonanywherewrapper. 100% API coverage Most of the codebase is documented

7 Dec 11, 2022
Campsite Reservation Cancellation Finder (Yellowstone National Park)

yellowstone-camping yellowstone-camping is a Campsite Reservation Cancellation Finder for Yellowstone National Park. This simple Python application wi

Justin Flannery 7 Aug 05, 2022