HTTP API for FGO game data. Transform the raw game data into something a bit more manageable.

View the API documentation here: https://api.atlasacademy.io.

If you are looking for only the type definitions and enums. You can download the fgo-api-types package.

• Configuration
  • Environment Variables
  • Secrets
• Development environment set up
• Run the API server
• Architecture
• Linting
• Formatting
• Dependencies
• Testing
• Helper scripts
  • format.ps1
  • extract_enums.py
  • update_ce_translation.py
  • load_rayshift_quest_list.py
  • get_test_data.py
  • niceexport.py

List of configuration variables for the main app. You can make a config.json file at the project root with the settings. Check the config.sample.json file for an example.

Required variables

• DATA: A JSON object with keys being region and values being gamedata's folder location and Postgresql DSN. Not all regions are required in the object. Any combination of regions is accepted.
• REDISDSN: Redis DSN to a Redis server for caching.

> cat .\secrets\rayshift_api_key
eca334a9-3289-4ad7-9b92-1ec2bbc3fc19
> cat .\secrets\redisdsn
redis://localhost

Make sure poetry is installed: https://python-poetry.org/docs/#installation.

Docker is recommended to set up the Postgres and redis servers but those can be set up manually as well. Postgres needs the PGroonga extension installed.

git clone --depth 1 --branch JP https://github.com/atlasacademy/fgo-game-data.git fgo-game-data-jp
git clone --depth 1 --branch NA https://github.com/atlasacademy/fgo-game-data.git fgo-game-data-na

# It's not neccecary to clone the other regions.
git clone --depth 1 --branch CN https://github.com/atlasacademy/fgo-game-data.git fgo-game-data-cn
git clone --depth 1 --branch KR https://github.com/atlasacademy/fgo-game-data.git fgo-game-data-kr
git clone --depth 1 --branch TW https://github.com/atlasacademy/fgo-game-data.git fgo-game-data-tw

git clone https://github.com/atlasacademy/fgo-game-data-api.git
cd fgo-game-data-api

# If you didn't clone other game data regions, remove them from the data field in config.json,
# and the services key in docker-compose.sample.yaml
cp config.sample.json config.json
cp docker-compose.sample.yml docker-compose.yml

docker-compose up -d

# Make sure you have the build prerequisites for psycopg2 installed if you are installing on Linux or macOS.
# https://www.psycopg.org/docs/install.html#build-prerequisites
# Debian/Ubuntu: sudo apt install libpq-dev python3-dev
# CentOS: sudo yum install python-devel postgresql-devel
poetry install
poetry shell

Run at the project root to start the API server:

> uvicorn app.main:app --reload --log-level debug --reload-dir app

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [16680] using watchgod
INFO      fgoapi: Loading game data …
INFO      fgoapi: Loaded game data in 15.14s.
INFO:     Started server process [33312]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
DEBUG     fgoapi: Processed in 0.21ms.
INFO:     127.0.0.1:56759 - "GET / HTTP/1.1" 307 Temporary Redirect
DEBUG     fgoapi: Processed in 0.24ms.
INFO:     127.0.0.1:56759 - "GET /rapidoc HTTP/1.1" 200 OK

Go to http://127.0.0.1:8000 for the API documentation.

Tips:

• Change write_postgres_data to false after the first run to speed up reloading if it's not needed (schema doesn't change or data hasn't changed).

• main.py: Main entrypoint of the application.
• routers/: Routers to deal with incoming requests. The routers call functions from core to get the response data.
• core/: Build response data. Get raw data from either db/helpers/ or redis/helpers/.
• data/: Import translation data into memory. Preprocess data to be imported into db and redis.
• db/: DB stuffs.
  • db/helpers/: Functions to be used by core to get data from Postgres.
• redis/: Redis stuffs.
  • redis/helpers/: Functions to be used by core to get data from Redis.
• schemas/: Response Pydantic models.
• models/: SQLAlchemy Core Tables.

ruff and mypy are used to lint the code. ruff's configuration and mypy's configuration are in pyproject.toml.

./scripts/lint.ps1

isort and black are used to format the code. isort's configuration is in pyproject.toml and black uses default settings. prettier is used to format the json files.

./scripts/format.ps1

Use poetry to manage the dependencies. Run poetry export after adding a production dependency.

poetry export -f requirements.txt -o requirements.txt

Run pytest at project root to run the tests or use coverage to get coverage statistics.

./scripts/test.ps1

Format all files with black, isort and prettier.

./scripts/format.ps1

Take the dump.cs generated by Il2CppDumper and write the gameenums.py file.

python scripts/extract_enums.py dump.cs_path app/schemas/gameenums.py

Update translation files with new NA CEs translations. --jp-master and --na-master arguments are not needed if environment variables JP_GAMEDATA and NA_GAMEDATA are set, added to the .env file or set in config.json.

python scripts/update_ce_translation.py --jp-master jp_master_path --na-master na_master_path

Update the rayshiftQuest tables with the list of available quests from Rayshift. This script should be run periodically to update the rayshiftQuest list.

python -m scripts.load_rayshift_quest_list

Run this script when the master data changed to update the tests or when new tests are added.

python -m tests.get_test_data --raw --nice --basic

Run this script to update the static export files in export/ folder.

python -m export.niceexport
./scripts/format.ps1