Dcf-game-infrastructure-public - Contains all the components necessary to run a DC finals (attack-defense CTF) game from OOO

Overview

dcf-game-infrastructure

All the components necessary to run a game of the OOO DC CTF finals.

Authors: adamd, hacopo, Erik Trickel, Zardus, and bboe

Design Philosophy

This repo contains all the game components necessary to run an Attack-Defense CTF that OOO used from 2018--2021.

The design is based on adamd's experience building the ictf-framework.

There are fundamental tenenats that we try to follow in the design of the system:

Spoke component model

The communication design of the components in the system (which you can kind of think of as micro-services) is a "spoke" model, where every component talks to the database (through a RESTish API), and no component directly talks to any other.

In this way, each component can be updated separately and can also be scaled independently using our k8s hosting.

This also made testing of each component easier, as the only dependence on a component is on the state of the database.

The only exception to this is the patchbot (the component that needs to test the patches submitted by the teams).

The database API puts the patchbot testing jobs into an RQ (Redis Queue), which all the patchbot workers pull jobs from.

Append-only database design

Fundamentally, a CTF database needs to calculate scores (that's essentially what the teams care about).

Prior design approaches that we've used would have a points or score column in the team table, and when they acquired or lost points, the app code would change this value.

However, many crazy things can happen during a CTF: recalculating scores or missed flags, even changing the scoring functions itself.

These can be difficult to handle depending on how the system is developed.

Therefore, we created a completely append-only database model, where no data in the DB is ever deleted or changed.

Even things like service status (the GOOD, OK, LOW, BAD that we used) is not a column in the services table. Every change of status would created a new StatusIndicator row, and the services would pull the latest version from this table.

Event model

Related to the append-only database design, everything in the database was represented by events.

The database would store all game events (in our game over the years was SLA_SCRIPT, FLAG_STOLEN, SET_FLAG, KOH_SCORE_FETCH, KOH_RANKING, PCAP_CREATED, PCAP_RELEASED, and STEALTH).

Then, the state of the game is based on these events.

An additional benefit is that these events could be shipped to the teams as part of the game_state.json.

Separate k8s clusters

How we ran this is with two k8s clusters: an admin cluster and a game cluster.

The admin cluster ran all of these components.

The game cluster ran all of the CTF challenges.

We used this design to do things like drop flags on the services. The flagbot used kubectl to drop a flag onto a service running in the other cluster.

This also allowed us to lock down the game cluster so that the vulnerable services couldn't make external requests, could be scaled separately, etc.

Install Requirements

This package is pip installable, and installs all dependencies. Do the following in a virtualenv:

$ pip install -e .

NOTE: If you want to connect to a mysql server (such as in prod or when deving against a mysql server), install the mysqlclient dependency like so:

$ pip install -e .[mysql]

Testing

Make sure the tests pass before you commit, and add new test cases in test for new features.

Note the database API now checks that the timezone is in UTC, so you'll need to specify that to run the tests:

$ TZ=UTC nosetests -v

Local Dev

If you're using tmux, I created a script local_dev.sh that will run a database-api, database-api frontend, team-interface backend, team-interface frontend, gamebot, and an ipython session with a database client created.

Just run the following

$ ./local_dev.sh

Deploy to prod

Build and -p push the image to production registry.

$ ./deploy.sh -p

Won't -r restart the running services, need to do:

$ ./deploy.sh -p -r

database-api

This has the tables for the database, a REST API to access it, and a python client to access the REST API.

See ooogame/database for details.

flagbot

Responsible for putting new flags into all the services for every game tick.

See ooogame/flagbot for details.

fresh-flagbot

Responsible for putting a new flags into a pod when it first comes up (from a team patching the service).

See ooogame/fresh_flagbot for details.

gamebot

Responsible for incrementing the game's ticks.

See ooogame/gamebot for details.

koh-scorebot

Responsible for extracting the King of the Hill (koh) scores from all the koh pods every tick, and submitting them to the database.

See ooogame/koh_scorebot for details.

team-interface

Responsible for providing an interface to the teams so that they can submit flags, get pcaps, upload patches, and get their patch status. Split into a backend flask REST API, which essentially wraps the database-api, and a React frontend.

See ooogame/team_interface for details.

pcapbot

Responsible for picking up all the newly generated pcaps, anonymize them, and if the service is releasing pcaps then release them.

See ooogame/pcapbot for details.

gamestatebot

Responsible for creating the game state at every new tick and storing them in the nfs, and release them publicly.

See ooogame/gamestatebot for details.

This is also the component that pushes data to the public scoreboard

Owner
Order of the Overflow
Order of the Overflow
Athena is the only tool that you will ever need to optimize your portfolio.

Athena Portfolio optimization is the process of selecting the best portfolio (asset distribution), out of the set of all portfolios being considered,

Indrajit 1 Mar 25, 2022
Source code of CIKM2021 Long Paper "PSSL: Self-supervised Learning for Personalized Search with Contrastive Sampling".

PSSL Source code of CIKM2021 Long Paper "PSSL: Self-supervised Learning for Personalized Search with Contrastive Sampling". It consists of the pre-tra

2 Dec 21, 2021
LoL Runes Recommender With Python

LoL-Runes-Recommender Para ejecutar la aplicación se debe llamar a execute_app.p

Sebastián Salinas 1 Jan 10, 2022
Image marine sea litter prediction Shiny

MARLITE Shiny app for floating marine litter detection in aerial images. This directory contains the instructions and software needed to install the S

19 Dec 22, 2022
NVIDIA Deep Learning Examples for Tensor Cores

NVIDIA Deep Learning Examples for Tensor Cores Introduction This repository provides State-of-the-Art Deep Learning examples that are easy to train an

NVIDIA Corporation 10k Dec 31, 2022
A full pipeline AutoML tool for tabular data

HyperGBM Doc | 中文 We Are Hiring! Dear folks,we are offering challenging opportunities located in Beijing for both professionals and students who are k

DataCanvas 240 Jan 03, 2023
Unofficial implementation of the paper: PonderNet: Learning to Ponder in TensorFlow

PonderNet-TensorFlow This is an Unofficial Implementation of the paper: PonderNet: Learning to Ponder in TensorFlow. Official PyTorch Implementation:

1 Oct 23, 2022
A Python library that enables ML teams to share, load, and transform data in a collaborative, flexible, and efficient way :chestnut:

Squirrel Core Share, load, and transform data in a collaborative, flexible, and efficient way What is Squirrel? Squirrel is a Python library that enab

Merantix Momentum 249 Dec 07, 2022
CCAFNet: Crossflow and Cross-scale Adaptive Fusion Network for Detecting Salient Objects in RGB-D Images

Code and result about CCAFNet(IEEE TMM) 'CCAFNet: Crossflow and Cross-scale Adaptive Fusion Network for Detecting Salient Objects in RGB-D Images' IEE

zyrant丶 14 Dec 29, 2021
Customer Segmentation using RFM

Customer-Segmentation-using-RFM İş Problemi Bir e-ticaret şirketi müşterilerini segmentlere ayırıp bu segmentlere göre pazarlama stratejileri belirlem

Nazli Sener 7 Dec 26, 2021
Python wrapper to access the amazon selling partner API

PYTHON-AMAZON-SP-API Amazon Selling-Partner API If you have questions, please join on slack Contributions very welcome! Installation pip install pytho

Michael Primke 330 Jan 06, 2023
JumpDiff: Non-parametric estimator for Jump-diffusion processes for Python

jumpdiff jumpdiff is a python library with non-parametric Nadaraya─Watson estimators to extract the parameters of jump-diffusion processes. With jumpd

Rydin 28 Dec 10, 2022
Neural Point-Based Graphics

Neural Point-Based Graphics Project   Video   Paper Neural Point-Based Graphics Kara-Ali Aliev1 Artem Sevastopolsky1,2 Maria Kolos1,2 Dmitry Ulyanov3

Ali Aliev 252 Dec 13, 2022
The personal repository of the work: *DanceNet3D: Music Based Dance Generation with Parametric Motion Transformer*.

DanceNet3D The personal repository of the work: DanceNet3D: Music Based Dance Generation with Parametric Motion Transformer. Dataset and Results Pleas

南嘉Nanga 36 Dec 21, 2022
Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow

Mask R-CNN for Object Detection and Segmentation This is an implementation of Mask R-CNN on Python 3, Keras, and TensorFlow. The model generates bound

Matterport, Inc 22.5k Jan 04, 2023
A time series processing library

Timeseria Timeseria is a time series processing library which aims at making it easy to handle time series data and to build statistical and machine l

Stefano Alberto Russo 11 Aug 08, 2022
RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation (CIKM'17)

RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation This is the implementation of RATE: Overcoming Noise and Spar

Yu Zhang 5 Feb 10, 2022
🚀 An end-to-end ML applications using PyTorch, W&B, FastAPI, Docker, Streamlit and Heroku

🚀 An end-to-end ML applications using PyTorch, W&B, FastAPI, Docker, Streamlit and Heroku

Made With ML 82 Jun 26, 2022
[CVPR'22] Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast

wseg Overview The Pytorch implementation of Weakly Supervised Semantic Segmentation by Pixel-to-Prototype Contrast. [arXiv] Though image-level weakly

Ye Du 96 Dec 30, 2022
Official implementation of "UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspective with Transformer"

[AAAI2022] UCTransNet This repo is the official implementation of "UCTransNet: Rethinking the Skip Connections in U-Net from a Channel-wise Perspectiv

Haonan Wang 199 Jan 03, 2023