PyJPBoatRace: Python-based Japanese boatrace tools ๐Ÿšค

Overview

PyJPBoatRace: Python-based Japanese boatrace tools ๐Ÿšค

GitHub top language GitHub tag (latest SemVer) GitHub GitHub last commit

Japanese boat race is extremely exciting sports. It is also fun to predict the results of races. Prediction like machine learning method requires data. Thus, this package provides you with useful tools for data analysis and auto-betting for boatrace.

Installation

Requirements

If you want to deposit, withdraw and betting with pyjpboatrace, one of the following browers is required at least:

  • Chrome
  • Firefox
  • Edge

Dependencies

  • python >= 3.7
  • requests>=2.25.0
  • beautifulsoup4>=4.9.3
  • selenium>=3.141.0
  • webdriver-manager>=3.2.2
  • msedge-selenium-tools

User installation

    pip install -U pyjpboatrace

How to use

  1. (optional) create an instance of UserInformation;
  2. (optional) create a selenium driver;
  3. create an instance of PyJPBoatrace;
  4. execute scraping and operating.

NOTE: you must create a UserInformation instance and a selenium driver to order to deposit, withdraw or bet.

UserInformation

  • pyjpboatrace.user_information.UserInformation(userid:str, pin:str, auth_pass:str, vote_pass:str, json_file:str)

NOTE: If you use a json file to create an instance of UserInformation, the json file should contain the following keys: userid, pin, auth_pass and vote_pass.

Selenium Driver

You can use the following functions to create selenium drivers:

  • pyjpboatrace.drivers.create_chrome_driver()
  • pyjpboatrace.drivers.create_firefox_driver()
  • pyjpboatrace.drivers.create_edge_driver()
  • pyjpboatrace.drivers.create_httpget_driver()

NOTE 1: you can use your own selenium driver.

NOTE 2: If you use create_httpget_driver, you cannot execute the following operations, deposit, withdraw or bet.

Scraping and Operating

PyJPBoatrace provides 2 main functions: scraping and operating.

The former is scraping race information, odds and so on; the latter is betting, depositing and withdrawing.

Scraping

  • Get a list of stadiums which hold races on the given day:

    • API:

      • PyJPBoatrace().get_stadiums(d: datetime.date) -> Dict[str, Any]
      • PyJPBoatrace().Stadiums.get(d: datetime.date) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2021-08-12",
        "ๅคงๆ‘":{
          "status":"-",
          "grade":[
            "ippan",
            "rookie"
          ],
          "timeframe":"nighter",
          "title":"ใƒซใƒผใ‚ญใƒผใ‚ทใƒชใƒผใ‚บ็ฌฌ๏ผ‘๏ผ–ๆˆฆใ€€ใ‚ชใƒผใƒซ้€ฒๅ…ฅๅ›บๅฎšใƒฌใƒผใ‚น",
          "period":[
            "2020-09-07",
            "2020-09-13"
          ],
          "day":"ๅˆๆ—ฅ"
        },
        ...
      }
  • To get 12 races held in the given stadium on the given day:

    • API:

      • PyJPBoatrace().get_12races(d: datetime.date, stadium: int) -> Dict[str, Any]
      • PyJPBoatrace().Races.get(d: datetime.date, stadium: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2021-08-12",
        "stadium": 1,
        "1R":{
            "vote_limit":"2020-01-01 10:00:00",
            "status":"็™บๅฃฒ็ต‚ไบ†",
            "racers":{
                "boat1":{
                    "name":"Name1",
                    "class":"A1"
                },
                "boat2":{
                    "name":"Name2",
                    "class":"A2"
                },
                "boat3":{
                    "name":"Name3",
                    "class":"B1"
                },
                "boat4":{
                    "name":"Name4",
                    "class":"B2"
                },
                "boat5":{
                    "name":"Name5",
                    "class":"A1"
                },
                "boat6":{
                    "name":"Name6",
                    "class":"B1"
                }
            }
        },
        ...,
        "12R":{
            "vote_limit":"2020-01-01 15:30:00",
            "status":"็™บๅฃฒ็ต‚ไบ†",
            "racers":{
                "boat1":{
                    "name":"Name1",
                    "class":"A1"
                },
                "boat2":{
                    "name":"Name2",
                    "class":"A2"
                },
                "boat3":{
                    "name":"Name3",
                    "class":"B1"
                },
                "boat4":{
                    "name":"Name4",
                    "class":"B2"
                },
                "boat5":{
                    "name":"Name5",
                    "class":"A1"
                },
                "boat6":{
                    "name":"Name6",
                    "class":"B1"
                }
            }
        }
      }
  • To get the basic information of the race in the stadium on a day:

    • API:

      • PyJPBoatrace().get_race_info(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
      • PyJPBoatrace().RaceInfo.get(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2021-08-12",
        "stadium": 10,
        "race": 1,
        "boat1": {
          "racerid": 9999,
          "class": "A1",
          "name": "Name1",
          "branch": "Somewhere",
          "birthplace": "Somewhere",
          "age": 40,
          "weight": 53.2,
          "F": 0,
          "L": 0,
          "aveST": 0.19,
          "global_win_pt": 6.43,
          "global_in2nd": 43.86,
          "global_in3rd": 68.42,
          "local_win_pt": 0,
          "local_in2nd": 0,
          "local_in3rd": 0,
          "motor": 42,
          "motor_in2nd": 35.48,
          "motor_in3rd": 56.13,
          "boat": 41,
          "boat_in2nd": 30.77,
          "boat_in3rd": 54.49,
          "result": [
            {
                "race": 8,
                "boat": 2,
                "course": 2,
                "ST": 0.24,
                "rank": 6
            },
            {},
            {
                "race": 4,
                "boat": 4,
                "course": 4,
                "ST": 0.28,
                "rank": 5
            },
            ...,
            {}
          ]
        },
        ...,
        "race_title": [
            "ใฟใใซใ‚ใ•ใ‚ฌใƒ",
            "1800m"
        ]
      }
  • To get the odds of win (ๅ˜ๅ‹) and place-show (่ค‡ๅ‹) of the race in the stadium on the day:

    • API:

      • PyJPBoatrace().get_odds_win_placeshow(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
      • PyJPBoatrace().WinPlaceshowOdds.get(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2020-10-24",
        "stadium": 14,
        "race": 1,
        "win":{
          "1": 1.0,
          "2": 6.8,
          "3": 9.3,
          "4": 41.3,
          "5": 36.1,
          "6": 72.3
        },
        "place_show":{
          "1": [1.0, 1.3],
          "2": [3.3, 5.0],
          "3": [1.5, 2.2],
          "4": [5.7, 8.9],
          "5": [1.1, 1.6],
          "6": [22.0, 33.3]
        },
        "update": "็ท ๅˆ‡ๆ™‚ใ‚ชใƒƒใ‚บ"
      }
  • To get the odds of quinella place (ๆ‹ก้€ฃ่ค‡) of the race in the stadioum on the day:

    • API:

      • PyJPBoatrace().get_odds_quinellaplace(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
      • PyJPBoatrace().QuinellaplaceOdds.get(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2020-10-24",
        "stadium": 14,
        "race": 1,
        "1=2": [1.2,1.4],
        "1=3": [1.3,1.8],
        ...,
        "5=6": [27.2,30.9],
        "update": "9:02"
      }
  • To get the odds of exacta (ไบŒ้€ฃๅ˜) and quinella (ไบŒ้€ฃ่ค‡) of the race in the stadioum on the day:

    • API:

      • PyJPBoatrace().get_odds_exacta_quinella(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
      • PyJPBoatrace().ExactaQuinellaOdds.get(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2020-10-24",
        "stadium": 14,
        "race": 1,
        "exacta":{
          "1-2": 2.5,
          "1-3": 2.8,
          ...,
          "6-5": 2931.0
        },
        "quinella":{
          "1=2": 3.0,
          "1=3": 2.1,
          ...,
          "5=6": 298.3
        },
        "update": "12:30"
      }
  • To get the odds of trifecta (ไธ‰้€ฃๅ˜) of the race in the stadioum on the day:

    • API:

      • PyJPBoatrace().get_odds_trifecta(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
      • PyJPBaotrace().TrifectaOdds.get(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2020-10-24",
        "stadium": 14,
        "race": 1,
        "1-2-3": 5.6,
        "1-2-4": 14.4,
        ...,
        "6-5-4": 8650.0,
        "update": "็ท ๅˆ‡ๆ™‚ใ‚ชใƒƒใ‚บ"
      }
  • To get the oods of trio ๏ผˆไธ‰้€ฃ่ค‡๏ผ‰ of the race in the stadioum on the day:

    • API:

      • PyJPBoatrace().get_odds_trio(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
      • PyJPBoatrace().Trio.get(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2020-10-24",
        "stadium": 14,
        "race": 1,
        "1=2=3": "ๆฌ ๅ ด",
        "1=2=4": "ๆฌ ๅ ด",
        "1=2=5": "ๆฌ ๅ ด",
        "1=2=6": "ๆฌ ๅ ด",
        "1=3=4": "ๆฌ ๅ ด",
        "2=3=4": 4.2,
        "1=3=5": "ๆฌ ๅ ด",
        "2=3=5": 30,
        "1=3=6": "ๆฌ ๅ ด",
        "2=3=6": 2.3,
        "1=4=5": "ๆฌ ๅ ด",
        "2=4=5": 25,
        "3=4=5": 79.2,
        "1=4=6": "ๆฌ ๅ ด",
        "2=4=6": 1.9,
        "3=4=6": 60,
        "1=5=6": "ๆฌ ๅ ด",
        "2=5=6": 26.7,
        "3=5=6": 132.1,
        "4=5=6": 90.1,
        "update": "15:30"
      }
  • To get the just-before information, e.g. weather and start-timing, of the race in the stadioum on the day:

    • API:

      • PyJPBoatrace().get_just_before_info(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
      • PyJPBoatrace().JustBeforeInfo.get(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2020-08-25",
        "stadium": 14,
        "race": 7,
        "boat1":{
            "name":"Name1",
            "weight":55.1,
            "weight_adjustment":0.0,
            "display_time":6.87,
            "tilt":0.0,
            "propeller":"",
            "parts_exchange":[
                "ใƒชใƒณใ‚ฐร—๏ผ‘",
                "ใ‚ทใƒฃใƒ•ใƒˆ"
            ],
            "previous_race":{}
        },
        ...,
        "boat6":{
            "name":"Name6",
            "weight":51.0,
            "weight_adjustment":0.0,
            "display_time":6.88,
            "tilt":-0.5,
            "propeller":"",
            "parts_exchange":[],
            "previous_race":{
                "race":3,
                "boat":3,
                "course":3,
                "ST":0.13,
                "rank":6
            }
        },
        "start_display":{
            "course1":{
                "boat":1,
                "ST":0.02
            },
            ...,
            "course6":{
                "boat":6,
                "ST":0.10
            }
        },
        "weather_information":{
            "direction":16,
            "weather":"ๆ™ด",
            "temperature":31.0,
            "wind_direction":14,
            "wind_speed":5,
            "water_temperature":27.0,
            "wave_height":5,
            "time":"6Rๆ™‚็‚น"
        }
      }
  • To get the race result of the race in the stadioum on the day:

    • API:

      • PyJPBoatrace().get_race_result(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
      • PyJPBoatrace().Result.get(d: datetime.date, stadium: int, race: int) -> Dict[str, Any]
    • Return:

      Sample
      {
        "date": "2020-10-24",
        "stadium": 14,
        "race": 1,
        "result":[
            {
                "rank":1,
                "boat":1,
                "name":"WHO1",
                "racerid":9999,
                "time":"1'50\"0"
            },
            ...,
            {
                "rank":6,
                "boat":2,
                "name":"WHO6",
                "racerid":8888,
                "time":""
            }
        ],
        "kimarite":"้€ƒใ’",
        "start_information":{
            "course1":{
                "boat":1,
                "ST":0.05
            },
            ...,
            "course6":{
                "boat":6,
                "ST":0.11
            }
        },
        "payoff":{
            "trifecta":{
                "result":"1-5-6",
                "payoff":12345,
                "popularity":34
            },
            ...,
            "quinella_place":[
                {
                    "result":"1=5",
                    "payoff":220,
                    "popularity":3
                },
                ...
            ],
            ...,
        },
        "weather_information":{
            "direction":16,
            "weather":"ๆ™ด",
            "temperature":17.0,
            "wind_direction":6,
            "wind_speed":5,
            "water_temperature":21.0,
            "wave_height":5
        },
        "return":[],
        "note":[]
      }

These functions return dict object.

Operations

  • To deposit money for betting (Unit: 1,000 yen):

    • API
      • PyJPBoatrace().deposit(num_of_thousands_yen: int) -> None
      • PyJPBoatrace().Deposit.do(num_of_thousands_yen: int) -> None
    • Return:
      • Nothing
  • To get the limit of betting amount, that is, your current deposit:

    • API
      • PyJPBoatrace().get_bet_limit() -> int
      • PyJPBoatrace().BettingLimitCheck.do() -> int
    • Return:
      • int: the amount of deposit
  • To withdraw your current deposit:

    • API:

      • PyJPBoatrace().withdraw() -> None
      • PyJPBoatrace().Widthdraw.do() -> None
    • Return:

      • Nothing
  • To bet some tickets.

    • API:

      • PyJPBoatrace().bet(
          stadium:int,
          race:int,
          trifecta_betting_dict: Dict[str, int],
          trio_betting_dict: Dict[str, int],
          exacta_betting_dict: Dict[str, int],
          quinela_betting_dict: Dict[str, int],
          quinellaplace_betting_dict: Dict[str, int],
          win_betting_dict: Dict[str, int],
          placeshow_betting_dict: Dict[str, int],
        ) -> bool
      • PyJPBoatrace().Bet.do(
          stadium:int,
          race:int,
          trifecta_betting_dict: Dict[str, int],
          trio_betting_dict: Dict[str, int],
          exacta_betting_dict: Dict[str, int],
          quinela_betting_dict: Dict[str, int],
          quinellaplace_betting_dict: Dict[str, int],
          win_betting_dict: Dict[str, int],
          placeshow_betting_dict: Dict[str, int],
        ) -> bool
    • Each dictionary consits of pairs of winning numbers and betting amount, e.g., {'1-2-3':100} for trifecta_betting_dict

IMPORTANT NOTE: you must give a driver other than HTTPGetDriver to use above actions.

Demo

Demo 1 : Getting odds data

The following example is useful. Suppose that you want get the odds of trifecta of 4th race in stadium "ๆก็”Ÿ" on 2020/12/02 and dump the result into data.json.

from datetime import date
import json
from pyjpboatrace import PyJPBoatrace

# initialize
boatrace_tools = PyJPBoatrace()

# get data
dic = boatrace_tools.get_odds_trifecta(d=date(2020,12,2), stadium=1, race=4)

# dump data
with open('data.json', 'w', encoding='utf-8') as f:
    json.dump(dic, f, ensure_ascii=False)

# close (you can use 'with' statement)
boatrace_tools.close()

You can get many kinds of data as this example.

Demo 2 : Betting

Suppose it is 2020/12/02 and you want to bet 200 yen on trifecta 1-3-4 and 100 yen on trio 1=3=4 in the 2nd race in stadium "ๆก็”Ÿ" on 2020/12/02. NOTE: you need google chrome in the following example.

from datetime import date
from pyjpboatrace import PyJPBoatrace
from pyjpboatrace.drivers import create_chrome_driver
from pyjpboatrace.const import STADIUMS_MAP
from pyjpboatrace.user_information import UserInformation

# initialize
user = UserInformation(
    userid='YOUR_USER_ID',
    pin='YOUR_PIN',
    auth_pass='YOUR_AUTHENTIFICATION_PASSWORD',
    vote_pass='YOUR_BETTING_PASSWORD',
)
boatrace_tools = PyJPBoatrace(
    driver=create_chrome_driver(),
    user_information=user
)

# deposit 1,000 yen
boatrace_tools.deposit(1)

# bet
stadium = {s:i for i,s in STADIUMS_MAP}.get("ๆก็”Ÿ")
race = 2
boatrace_tools.bet(
    stadium=stadium,
    race=race,
    trifecta_betting_dict={'1-3-4':200},
    trio_betting_dict={'1=3=4':100}
)

# waiting for the race result ...

# withdraw
boatrace_tools.withdraw()

# close (you can use 'with' statement)
boatrace_tools.close()

NOTE

The map between integers and stadiums is given by STADIUMS_MAP in pyjpboatrace.const.

Contribution Guide

Requirement

  • Python >= 3.7
  • Pipenv (You can install pipenv by pip install pipenv)
  • Chrome
  • Firefox
  • bash
  • curl

Issues

  • For any bugs, use BUG REPORT to create an issue.

  • For any enhancement, use FEATURE REQUEST to create an issue.

  • For other topics, create an issue with a clear and concise description.

Pull Request

  1. Fork (https://github.com/hmasdev/pyjpboatrace/fork);
  2. Create your feature branch (git checkout -b feautre/xxxx);
  3. Test codes according to Test Subsection;
  4. Commit your changes (git commit -am 'Add xxxx feature);
  5. Push to the branch (git push origin feature/xxxx);
  6. Create new Pull Request

Test

You can do unit tests and integration tests as follows:

$ ./download_html_for_test.sh  # Only 1 time
$ pipenv run pytest -m "not integrate and not spending_money" # unit tests
$ pipenv run pytest  # unit tests and integration tests

pipenv run pytest does not test depositing, withdrawing or betting. If you want to test them, make .secrets.json at first:

{
  "userid": "YOUR_USER_ID",
  "pin": "YOUR_PIN",
  "auth_pass": "YOUR_AUTHENTIFICATION_PASSWORD",
  "vote_pass": "YOUR_BETTING_PASSWORD"
}

Then, run

$ pipenv run pytest -m "spending_money"

WARNING: Tests with spending_money spend 700 yen.

LICENSE

MIT

Authors

hmasdev

Comments
  • [BUG] Scheduled tests failed for stadiums, race_info and just_before_info scrapers

    [BUG] Scheduled tests failed for stadiums, race_info and just_before_info scrapers

    Describe the bug

    Pytest failed.

    Environment

    • github actions (See https://github.com/hmasdev/pyjpboatrace/actions/runs/1766385345)
    • Python Version: 3.7, 3.8 and 3.9

    To Reproduce

    I have confirmed that the following test fails in my environment as well:

    $ pipenv install --dev
    $ ./download_html_for_test.sh
    $ pipenv run pytest
    

    Screenshots or codes

    See https://github.com/hmasdev/pyjpboatrace/actions/runs/1938733501

    Expected behavior

    The test should pass.

    Problems

    Reasons (Optional)

    TBD

    bug 
    opened by hmasdev 3
  • [BUG] PyPI workflow in actions does not work

    [BUG] PyPI workflow in actions does not work

    Describe the bug

    PyPI workflow in actions, which runs when released, returns exit code 1 during uploading .whl and .tar.gz with twine.

    https://github.com/hmasdev/pyjpboatrace/actions/runs/1103843756

    Environment

    To Reproduce (Probabily)

    Steps to reproduce the behavior:

    1. Go to https://github.com/hmasdev/pyjpboatrace/releases
    2. Click on "Draft a new release"
    3. Input tag version, title and description.
    4. Click on "Publish release", which triggers PyPI workflow.
    5. Error during PyPI workflow in Upload step as follows:
      Traceback (most recent call last):
         File "/home/runner/.local/share/virtualenvs/pyjpboatrace-cg8vEFYy/bin/twine", line 5, in <module>
           from twine.__main__ import main
         File "/home/runner/.local/share/virtualenvs/pyjpboatrace-cg8vEFYy/lib/python3.8/site-packages/twine/__init__.py", line 27, in <module>
           import importlib_metadata
      ModuleNotFoundError: No module named 'importlib_metadata'
      Error: Process completed with exit code 1.
      

    Expected behavior

    Twine succeeds in uploading .whl and .tar.gz to pypi server.

    Uploading distributions to https://upload.pypi.org/legacy/
    Uploading pyjpboatrace-0.2.0-py3-none-any.whl
    100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 63.5k/63.5k [00:02<00:00, 24.2kB/s]
    Uploading pyjpboatrace-0.2.0.tar.gz
    100%|โ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆโ–ˆ| 50.3k/50.3k [00:01<00:00, 31.2kB/s] 
    
    View at:
    https://pypi.org/project/pyjpboatrace/0.2.0/
    

    Problems

    If this bug is not fixed, we can not upload this package to pypi automatically, that is, manual operation is required when uploading.

    Reasons (Optional)

    Checking the log, I found pipenv uses python 3.8.10 as follows:

    Creating a virtualenv for this project...
    Pipfile: /home/runner/work/pyjpboatrace/pyjpboatrace/Pipfile
    Using /usr/bin/python3.8 (3.8.10) to create virtualenv...
    created virtual environment CPython3.8.10.final.0-64 in 694ms
    

    This is probably a reason for this bug.

    See https://stackoverflow.com/a/59734959/16567832

    bug 
    opened by hmasdev 2
  • [BUG] If grade is SG, parsing fails.

    [BUG] If grade is SG, parsing fails.

    If grade is SG, parsing fails. https://www.boatrace.jp/owpc/pc/race/index?hd=20221030

    from datetime import date
    import json
    from pyjpboatrace import PyJPBoatrace
    boatrace_tools = PyJPBoatrace()
    dic = boatrace_tools.get_stadiums(d=date(2022,10,30))
    with open('data.json', 'w', encoding='utf-8') as f:
        json.dump(dic, f, ensure_ascii=False)
    boatrace_tools.close()
    

    ... 'Always': {'status': '-', 'grade': ['[FailedToParse]is-SGa'], 'timeframe': '', 'title': 'The 69th Boat Race Derby', 'period': ['2022-10-25', '2022-10-30'], 'day': 'Last day'}, ...

    bug 
    opened by miyamamoto 1
  • Add timestamp to the responses from odds scrapers

    Add timestamp to the responses from odds scrapers

    Abstract

    The response dictionaries from odds scrapers, like get_odds_trifecta (TrifectaOddsScraper.get), should contain the timestamp when odds number are updated.

    AsIs: Background

    Odds scrapers for a race return the final odds after the race is over. However, they return uncertain odds before the race is not over.

    The numbers varies depending on when the scrapers are called. For example, get_odds_trifecta returns one-min-ahead trifecta odds when you call get_odds_trifecta one minute before the deadline. On the other hand, it returns five-min-ahead trifecta odds when you call get_odds_trifecta five minutes before the deadline. The problem is that you can not find the timestamp when odds have been updated in the reponse from the scraper.

    The following is an example:

    from datetime import date, datetime
    import json
    from pyjpboatrace import PyJPBoatrace
    
    # initialize
    boatrace_tools = PyJPBoatrace()
    
    # get data
    d=date(2020,12,2)
    stadium=1
    race=4
    timestamp = datetime.now()
    response = boatrace_tools.get_odds_trifecta(d=d, stadium=stadium, race=race)
    
    # add timestamp to response
    # because response is a dictionary like {"1-2-3": 7.0, ..., "6-5-4": 9999}
    response["timestamp"] = timestamp
    
    # close (you can use 'with' statement)
    boatrace_tools.close()
    

    ToBe: Describe the solution you'd like

    I want odds scrapers to response the timestamp in addition to odds values. Altanatively, I want odds scrapers to scrape the update timestamp from the target website, because odds pages in pyjpboatrace.jp contain a timestamp when the odds are updated.

    enhancement 
    opened by hmasdev 1
  • Add arguments (date, stadium and race) to response dictionary

    Add arguments (date, stadium and race) to response dictionary

    Abstract

    The response dictionaries from scraping functions e.g. get_stadiums (StadiumsScraper.get) and get_odds_trifecta (TrifectaOddsScraper.get), should contain the arguments: date, stadium number and race number.

    AsIs: Background

    A use case of pyjpboatrace is to use pyjpboatrace scrapers to create boatrace database, where date, stadium number and race number are likely to be primary keys. However, the response dictionaries from scraping functions don't contain the arguments, so it is required to add the arguments, like date, stadium number and race number, to the response dictionaries after scraping.

    The following is an example:

    from datetime import date
    import json
    from pyjpboatrace import PyJPBoatrace
    
    # initialize
    boatrace_tools = PyJPBoatrace()
    
    # get data
    d=date(2020,12,2)
    stadium=1
    race=4
    response = boatrace_tools.get_odds_trifecta(d=d, stadium=stadium, race=race)
    
    # add date, stadium and race to response
    # because response is a dictionary like {"1-2-3": 7.0, ..., "6-5-4": 9999}
    response["date"] = d
    response["stadium"] = stadium
    response["race"] = race
    
    # close (you can use 'with' statement)
    boatrace_tools.close()
    

    ToBe: Describe the solution you'd like

    I want the response dictionaries from pyjpboatrace's scrapers to contain the arugments like date, stadium number and race number. For example, the reponse from get_odds_trifecta should be like

    {
      "date": date(2020, 1, 1),  # This is a new element.
      "stadium": 1,    # This is a new element.
      "race": 1,    # This is a new element.
      "1-2-3": 5.6,
      "1-2-4": 14.4,
      ...,
      "6-5-4": 8650.0
    }
    
    enhancement 
    opened by hmasdev 1
  • ใ™ในใฆใฎ้ธๆ‰‹ใŒใƒ•ใƒฉใ‚คใƒณใ‚ฐใ ใฃใŸ้š›ใฎget_race_resultใฎใ‚จใƒฉใƒผใซใคใ„ใฆ

    ใ™ในใฆใฎ้ธๆ‰‹ใŒใƒ•ใƒฉใ‚คใƒณใ‚ฐใ ใฃใŸ้š›ใฎget_race_resultใฎใ‚จใƒฉใƒผใซใคใ„ใฆ

    ใ™ในใฆใฎ้ธๆ‰‹ใŒใƒ•ใƒฉใ‚คใƒณใ‚ฐใ ใฃใŸ้š›ใซใŠใ„ใฆget_race_resultใŒใ‚จใƒฉใƒผใ‚’ๅใๆง˜ๅญใงใ™ใ€‚ https://www.boatrace.jp/owpc/pc/race/raceresult?rno=3&jcd=21&hd=20180101

    dic = boatrace_tools.get_race_result(d=date(2018,1,1), stadium=21, race=3)
    
    bug 
    opened by miyamamoto 1
  • Feature/update readme

    Feature/update readme

    Abstract

    Update output samples and add a note of get_race_result in README.md

    Objective

    • output samples are outdated;

    Type of change

    • Documentation update

    What you did

    • transformed output samples to python codes;
    • added a note of get_race_result
    opened by hmasdev 0
  • fix a bug in parse_html_index during parsing SG

    fix a bug in parse_html_index during parsing SG

    Abstract

    Fix codes for #64

    Objective

    To fix parsers even when grade is SG.

    Type of change

    • Bug fix (non-breaking change which fixes an issue)

    What you did

    • Fixed parsers.

    Impacts

    • When using PyJPBoatrace.get_stadiums, the grade is correctly parsed even when the grade is SG.
      • AsIs: the grade is parsed as "[FailedToParse]is-SGa" when the grade is SG;
      • ToBe: the grade is parsed as "SG" when the grade is SG.
    opened by hmasdev 0
  • Feature/add test case for multiple results

    Feature/add test case for multiple results

    Abstract

    Add test cases for #60

    Objective

    To test the api to get the result of a race where a tie occurred.

    Type of change

    • [o] Bug fix (non-breaking change which fixes an issue)

    What you did

    • [o] what you did;
      • add tests
    opened by hmasdev 0
  • Update setup.cfg

    Update setup.cfg

    Abstract

    Update setup.cfg with adding python 3.10 to classifiers

    Objective

    To indicate that it can be used in a Python 3.10 environment.

    Type of change

    Update setup.cfg

    What you did

    • Add python 3.10 to classifiers

    Operation check

    • To check whether python 3.10 is added appropriately to classifiers
    opened by hmasdev 0
  • add python 3.10 to pytest.yaml

    add python 3.10 to pytest.yaml

    Abstract

    Add test with python 3.10 to pytest.yaml

    Objective

    To test pyjpboatrace with python 3.10 in addition to python 3.7, python 3.8 and python 3.9.

    Type of change

    Update github actions

    What you did

    • Add test with python 3.10 to pytest.yaml

    Operation check

    • To check whether the tests in github actions pass
    opened by hmasdev 0
  • Request to automate updates to README.md.

    Request to automate updates to README.md.

    Abstract

    Request to automate updates to README.md.

    AsIs: Background

    The current README.md contains the PyJPBoatrace API specifications and sample output. However, when changes are made to the code, especially when there are updates to the output of each API, the sample output must be obtained manually and the README.md updated.

    ToBe: Describe the solution you'd like

    If there are updates to the output of each API, the sample output updates in README.md are automatically committed to the main branch when the corresponding pull request is merged into the main branch.

    Additional context

    • Step.1: to add a script to use the current code to update sample outputs;
    • Step.2: to create a workflow to execute the script above to update README.md and commit the change to main.

    MEMO

    This problem because apparent in https://github.com/hmasdev/pyjpboatrace/pull/61#issuecomment-1304954123

    enhancement 
    opened by hmasdev 0
Releases(v0.2.3a)
  • v0.2.3a(Dec 4, 2022)

    What's Changed

    • add python 3.10 to pytest.yaml by @hmasdev in https://github.com/hmasdev/pyjpboatrace/pull/58
    • Update setup.cfg by @hmasdev in https://github.com/hmasdev/pyjpboatrace/pull/59
    • Fixed to take multiple results when ties exist by @miyamamoto in https://github.com/hmasdev/pyjpboatrace/pull/61
    • Feature/add test case for multiple results by @hmasdev in https://github.com/hmasdev/pyjpboatrace/pull/62
    • fix a bug in parse_html_index during parsing SG by @hmasdev in https://github.com/hmasdev/pyjpboatrace/pull/65
    • Feature/update readme by @hmasdev in https://github.com/hmasdev/pyjpboatrace/pull/66

    New Contributors

    • @miyamamoto made their first contribution in https://github.com/hmasdev/pyjpboatrace/pull/61

    Full Changelog: https://github.com/hmasdev/pyjpboatrace/compare/v0.2.2a...v0.2.3a

    Source code(tar.gz)
    Source code(zip)
  • v0.2.2a(Mar 9, 2022)

  • v0.2.1a(Aug 12, 2021)

    • Add the timestamp when odds are updated to responses from odds scraper, like PyJPBoatrace.get_odds_trifecta and PyJPBoatrace.TrifectaOdds.get;
    • Add date, stadium and race to responses.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0a(Aug 6, 2021)

  • v0.1.0a(Dec 28, 2020)

  • v0.0.0a(Dec 5, 2020)

Data and evaluation code for the paper WikiNEuRal: Combined Neural and Knowledge-based Silver Data Creation for Multilingual NER (EMNLP 2021).

Data and evaluation code for the paper WikiNEuRal: Combined Neural and Knowledge-based Silver Data Creation for Multilingual NER. @inproceedings{tedes

Babelscape 40 Dec 11, 2022
This project uses word frequency and Term Frequency-Inverse Document Frequency to summarize a text.

Text Summarizer This project uses word frequency and Term Frequency-Inverse Document Frequency to summarize a text. Team Members This mini-project was

1 Nov 16, 2021
Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

T5: Text-To-Text Transfer Transformer The t5 library serves primarily as code for reproducing the experiments in Exploring the Limits of Transfer Lear

Google Research 4.6k Jan 01, 2023
OpenAI CLIP text encoders for multiple languages!

Multilingual-CLIP OpenAI CLIP text encoders for any language Colab Notebook ยท Pre-trained Models ยท Report Bug Overview OpenAI recently released the pa

Fredrik Carlsson 481 Dec 30, 2022
Reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer: Self-Attention with Linear Complexity)

Linear Multihead Attention (Linformer) PyTorch Implementation of reproducing the Linear Multihead Attention introduced in Linformer paper (Linformer:

Kui Xu 58 Dec 23, 2022
CATs: Semantic Correspondence with Transformers

CATs: Semantic Correspondence with Transformers For more information, check out the paper on [arXiv]. Training with different backbones and evaluation

74 Dec 10, 2021
Nmt - TensorFlow Neural Machine Translation Tutorial

Neural Machine Translation (seq2seq) Tutorial Authors: Thang Luong, Eugene Brevdo, Rui Zhao (Google Research Blogpost, Github) This version of the tut

6.1k Dec 29, 2022
Guide: Finetune GPT2-XL (1.5 Billion Parameters) and GPT-NEO (2.7 B) on a single 16 GB VRAM V100 Google Cloud instance with Huggingface Transformers using DeepSpeed

Guide: Finetune GPT2-XL (1.5 Billion Parameters) and GPT-NEO (2.7 Billion Parameters) on a single 16 GB VRAM V100 Google Cloud instance with Huggingfa

289 Jan 06, 2023
Transformer training code for sequential tasks

Sequential Transformer This is a code for training Transformers on sequential tasks such as language modeling. Unlike the original Transformer archite

Meta Research 578 Dec 13, 2022
A Multi-modal Model Chinese Spell Checker Released on ACL2021.

ReaLiSe ReaLiSe is a multi-modal Chinese spell checking model. This the office code for the paper Read, Listen, and See: Leveraging Multimodal Informa

DaDa 106 Dec 29, 2022
Named-entity recognition using neural networks. Easy-to-use and state-of-the-art results.

NeuroNER NeuroNER is a program that performs named-entity recognition (NER). Website: neuroner.com. This page gives step-by-step instructions to insta

Franck Dernoncourt 1.6k Dec 27, 2022
The following links explain a bit the idea of semantic search and how search mechanisms work by doing retrieve and rerank

Main Idea The following links explain a bit the idea of semantic search and how search mechanisms work by doing retrieve and rerank Semantic Search Re

Sergio Arnaud Gomez 2 Jan 28, 2022
Exploring dimension-reduced embeddings

sleepwalk Exploring dimension-reduced embeddings This is the code repository. See here for the Sleepwalk web page. License and disclaimer This program

S. Anders's research group at ZMBH 91 Nov 29, 2022
Behavioral Testing of Clinical NLP Models

Behavioral Testing of Clinical NLP Models This repository contains code for testing the behavior of clinical prediction models based on patient letter

Betty van Aken 2 Sep 20, 2022
NeMo: a toolkit for conversational AI

NVIDIA NeMo Introduction NeMo is a toolkit for creating Conversational AI applications. NeMo product page. Introductory video. The toolkit comes with

NVIDIA Corporation 5.3k Jan 04, 2023
Code for using and evaluating SpanBERT.

SpanBERT This repository contains code and models for the paper: SpanBERT: Improving Pre-training by Representing and Predicting Spans. If you prefer

Meta Research 798 Dec 30, 2022
CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Generation

CPT This repository contains code and checkpoints for CPT. CPT: A Pre-Trained Unbalanced Transformer for Both Chinese Language Understanding and Gener

fastNLP 342 Jan 05, 2023
Mkdocs + material + cool stuff

Modern-Python-Doc-Example mkdocs + material + cool stuff Doc is live here Features out of the box amazing good looking website thanks to mkdocs.org an

Francesco Saverio Zuppichini 61 Oct 26, 2022
Training code of Spatial Time Memory Network. Semi-supervised video object segmentation.

Training-code-of-STM This repository fully reproduces Space-Time Memory Networks Performance on Davis17 val set&Weights backbone training stage traini

haochen wang 128 Dec 11, 2022
Natural Language Processing

NLP Natural Language Processing apps Multilingual_NLP.py start #This script is demonstartion of Mul

Ritesh Sharma 1 Oct 31, 2021