Skip to content

Openmesh-Network/openmesh-collectors

Repository files navigation

Openmesh Collectors

Build Status

image

Openmesh collectors turns streams from many data sources into a single unified format to be used internally.

Currently the main data sources are Web3 financial data.

Dependencies

  • Docker
  • Python

Libraries

  • Apache Kafka

How to run

All the information for running collectors locally is in our contributor guide. It'll show you how to set up the environment and help you better understand how you can contribute to the project.

Supported Data Sources

For off-chain data, the following exchanges are supported:

  • Binance
  • Binance Futures
  • Bitfinex
  • Bybit
  • Coinbase
  • Deribit
  • Gemini
  • Kraken
  • Kraken Futures
  • Phemex

For on-chain data, the following DeFi protocols are supported:

Ethereum

  • Curve Finance
  • Dodoex
  • Hashflow
  • Sushiswap
  • Uniswap V2
  • Uniswap V3

Free Managed Service

Openmesh is also hosted as a free managed service where you can tap into crypto market data via our Websocket API, or analyze historical data via our query tool. View the documentation here.

Examples of Data

Binance Trade

{
    "exchange": "binance",
    "symbol": "BTC.USDT",
    "price": "18200.46",
    "size": "0.00987",
    "taker_side": "sell",
    "trade_id": "2465019970",
    "maker_order_id": "17236243366",
    "taker_order_id": "17236243463",
    "event_timestamp": "2023-01-12T11:44:40.270000Z",
    "atom_timestamp": 1673523880325488
}

Uniswap V2 Swap

{
    "exchange": "uniswap-v2",
    "atomTimestamp": 1673523937320352,
    "blockTimestamp": "2023-01-12T11:45:35Z",
    "maker": null,
    "taker": "0xD6f3E785D2C8Fd698A2B341DDC860f291aB7fF61",
    "tokenBought": "DAI",
    "tokenSold": "WETH",
    "tokenBoughtAddr": "0x6b175474e89094c44da98b954eedeac495271d0f",
    "tokenSoldAddr": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2",
    "amountBought": "83.629106541068626336000000",
    "amountSold": "0.060000000000000000000000",
    "logIndex": 163,
    "transactionHash": "0x098b29872b5a2bc4aaf93fc7cbbdaced677657c051338b54494352d9cffe1fcb",
    "blockHash": "0xbcde35a84ed4985e8eafbc8409cda523268d1eece33b8ec5e7195a53897c66d4",
    "blockNumber": 16390510,
    "pairAddr": "0xa478c2975ab1ea89e8196811f51a7b7ade33eb11"
}

Ethereum Block

{
    "atomTimestamp": 1673523998673950,
    "number": 16390515,
    "hash": "0xc874e1704208666e624402f876cf2c0e6ef770f3135b259bb30508d0706942b3",
    "parentHash": "0x46a16ce33a2ba476762212e418171f43bf083f6060e3fc74efc5ab46eb619ce3",
    "nonce": "0x0000000000000000",
    "sha3Uncles": "0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347",
    "logsBloom": "0x00200050a1000000121806008900000202024020000008a020010200040004140001092044081420800018000018010052000040088028002000000010a02000014000000020000828020048110000284094000068400000000004008002a20012040800020000010040800000000809c02000000080441041808011a2088820100003140080180004480000001010008000828129048208402000c0001200000200054240006201200000c40002100800200000203001801020000000880000a1009006000108100002200000000000001102400000005000000902000020001030b02a04400100000112800000080002120000400040400000004048001000",
    "transactionsRoot": "0xb5801367d4d95058a24ff7aec1f5711e01cb8d4130c88eda7444d0b685e751c6",
    "stateRoot": "0x8fa22d1026db5f7f615392cc83290ea46c91b8a72e75a9397dee77ad88d40b52",
    "receiptsRoot": "0xd19d05a65e76d2133af12d7d32cbec58ffe8616924ead5c7c7571e3d07aafd73",
    "miner": "0x2f9fc79066bb3ffb072bcf4e26b635436d026ce5",
    "difficulty": 0,
    "totalDifficulty": "58750003716598355984384",
    "extraData": "0x",
    "size": 6491,
    "gasLimit": "30000000",
    "gasUsed": "1755498",
    "blockTimestamp": "2023-01-12T11:46:35Z"
}

About

Data collection, enhancement, and metrics calculation.

Resources

License

Code of conduct

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published