Dcf-game-infrastructure-public - Contains all the components necessary to run a DC finals (attack-defense CTF) game from OOO

Overview

dcf-game-infrastructure

All the components necessary to run a game of the OOO DC CTF finals.

Authors: adamd, hacopo, Erik Trickel, Zardus, and bboe

Design Philosophy

This repo contains all the game components necessary to run an Attack-Defense CTF that OOO used from 2018--2021.

The design is based on adamd's experience building the ictf-framework.

There are fundamental tenenats that we try to follow in the design of the system:

Spoke component model

The communication design of the components in the system (which you can kind of think of as micro-services) is a "spoke" model, where every component talks to the database (through a RESTish API), and no component directly talks to any other.

In this way, each component can be updated separately and can also be scaled independently using our k8s hosting.

This also made testing of each component easier, as the only dependence on a component is on the state of the database.

The only exception to this is the patchbot (the component that needs to test the patches submitted by the teams).

The database API puts the patchbot testing jobs into an RQ (Redis Queue), which all the patchbot workers pull jobs from.

Append-only database design

Fundamentally, a CTF database needs to calculate scores (that's essentially what the teams care about).

Prior design approaches that we've used would have a points or score column in the team table, and when they acquired or lost points, the app code would change this value.

However, many crazy things can happen during a CTF: recalculating scores or missed flags, even changing the scoring functions itself.

These can be difficult to handle depending on how the system is developed.

Therefore, we created a completely append-only database model, where no data in the DB is ever deleted or changed.

Even things like service status (the GOOD, OK, LOW, BAD that we used) is not a column in the services table. Every change of status would created a new StatusIndicator row, and the services would pull the latest version from this table.

Event model

Related to the append-only database design, everything in the database was represented by events.

The database would store all game events (in our game over the years was SLA_SCRIPT, FLAG_STOLEN, SET_FLAG, KOH_SCORE_FETCH, KOH_RANKING, PCAP_CREATED, PCAP_RELEASED, and STEALTH).

Then, the state of the game is based on these events.

An additional benefit is that these events could be shipped to the teams as part of the game_state.json.

Separate k8s clusters

How we ran this is with two k8s clusters: an admin cluster and a game cluster.

The admin cluster ran all of these components.

The game cluster ran all of the CTF challenges.

We used this design to do things like drop flags on the services. The flagbot used kubectl to drop a flag onto a service running in the other cluster.

This also allowed us to lock down the game cluster so that the vulnerable services couldn't make external requests, could be scaled separately, etc.

Install Requirements

This package is pip installable, and installs all dependencies. Do the following in a virtualenv:

$ pip install -e .

NOTE: If you want to connect to a mysql server (such as in prod or when deving against a mysql server), install the mysqlclient dependency like so:

$ pip install -e .[mysql]

Testing

Make sure the tests pass before you commit, and add new test cases in test for new features.

Note the database API now checks that the timezone is in UTC, so you'll need to specify that to run the tests:

$ TZ=UTC nosetests -v

Local Dev

If you're using tmux, I created a script local_dev.sh that will run a database-api, database-api frontend, team-interface backend, team-interface frontend, gamebot, and an ipython session with a database client created.

Just run the following

$ ./local_dev.sh

Deploy to prod

Build and -p push the image to production registry.

$ ./deploy.sh -p

Won't -r restart the running services, need to do:

$ ./deploy.sh -p -r

database-api

This has the tables for the database, a REST API to access it, and a python client to access the REST API.

See ooogame/database for details.

flagbot

Responsible for putting new flags into all the services for every game tick.

See ooogame/flagbot for details.

fresh-flagbot

Responsible for putting a new flags into a pod when it first comes up (from a team patching the service).

See ooogame/fresh_flagbot for details.

gamebot

Responsible for incrementing the game's ticks.

See ooogame/gamebot for details.

koh-scorebot

Responsible for extracting the King of the Hill (koh) scores from all the koh pods every tick, and submitting them to the database.

See ooogame/koh_scorebot for details.

team-interface

Responsible for providing an interface to the teams so that they can submit flags, get pcaps, upload patches, and get their patch status. Split into a backend flask REST API, which essentially wraps the database-api, and a React frontend.

See ooogame/team_interface for details.

pcapbot

Responsible for picking up all the newly generated pcaps, anonymize them, and if the service is releasing pcaps then release them.

See ooogame/pcapbot for details.

gamestatebot

Responsible for creating the game state at every new tick and storing them in the nfs, and release them publicly.

See ooogame/gamestatebot for details.

This is also the component that pushes data to the public scoreboard

Owner
Order of the Overflow
Order of the Overflow
Escaping the Gradient Vanishing: Periodic Alternatives of Softmax in Attention Mechanism

Period-alternatives-of-Softmax Experimental Demo for our paper 'Escaping the Gradient Vanishing: Periodic Alternatives of Softmax in Attention Mechani

slwang9353 0 Sep 06, 2021
A package for music online and offline rhythmic information analysis including music Beat, downbeat, tempo and meter tracking.

BeatNet A package for music online and offline rhythmic information analysis including music Beat, downbeat, tempo and meter tracking. This repository

Mojtaba Heydari 157 Dec 27, 2022
OpenFace – a state-of-the art tool intended for facial landmark detection, head pose estimation, facial action unit recognition, and eye-gaze estimation.

OpenFace 2.2.0: a facial behavior analysis toolkit Over the past few years, there has been an increased interest in automatic facial behavior analysis

Tadas Baltrusaitis 5.8k Dec 31, 2022
GDR-Net: Geometry-Guided Direct Regression Network for Monocular 6D Object Pose Estimation. (CVPR 2021)

GDR-Net This repo provides the PyTorch implementation of the work: Gu Wang, Fabian Manhardt, Federico Tombari, Xiangyang Ji. GDR-Net: Geometry-Guided

169 Jan 07, 2023
Code for PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning

PackNet: https://arxiv.org/abs/1711.05769 Pretrained models are available here: https://uofi.box.com/s/zap2p03tnst9dfisad4u0sfupc0y1fxt Datasets in Py

Arun Mallya 216 Jan 05, 2023
CaFM-pytorch ICCV ACCEPT Introduction of dataset VSD4K

CaFM-pytorch ICCV ACCEPT Introduction of dataset VSD4K Our dataset VSD4K includes 6 popular categories: game, sport, dance, vlog, interview and city.

96 Jul 05, 2022
Code for How To Create A Fully Automated AI Based Trading System With Python

AI Based Trading System This code works as a boilerplate for an AI based trading system with yfinance as data source and RobinHood or Alpaca as broker

Rubén 196 Jan 05, 2023
A python/pytorch utility library

A python/pytorch utility library

Jiaqi Gu 5 Dec 02, 2022
Chainer Implementation of Fully Convolutional Networks. (Training code to reproduce the original result is available.)

fcn - Fully Convolutional Networks Chainer implementation of Fully Convolutional Networks. Installation pip install fcn Inference Inference is done as

Kentaro Wada 218 Oct 27, 2022
Versatile Generative Language Model

Versatile Generative Language Model This is the implementation of the paper: Exploring Versatile Generative Language Model Via Parameter-Efficient Tra

Zhaojiang Lin 17 Dec 02, 2022
A general-purpose programming language, focused on simplicity, safety and stability.

The Rivet programming language A general-purpose programming language, focused on simplicity, safety and stability. Rivet's goal is to be a very power

The Rivet programming language 17 Dec 29, 2022
A denoising diffusion probabilistic model synthesises galaxies that are qualitatively and physically indistinguishable from the real thing.

Realistic galaxy simulation via score-based generative models Official code for 'Realistic galaxy simulation via score-based generative models'. We us

Michael Smith 32 Dec 20, 2022
Plug-n-Play Reinforcement Learning in Python with OpenAI Gym and JAX

coax is built on top of JAX, but it doesn't have an explicit dependence on the jax python package. The reason is that your version of jaxlib will depend on your CUDA version.

128 Dec 27, 2022
Codes for SIGIR'22 Paper 'On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation'

OD-Rec Codes for SIGIR'22 Paper 'On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation' Paper, saved teacher models and Andro

Xin Xia 11 Nov 22, 2022
Facilitates implementing deep neural-network backbones, data augmentations

Introduction Nowadays, the training of Deep Learning models is fragmented and unified. When AI engineers face up with one specific task, the common wa

40 Dec 29, 2022
An End-to-End Machine Learning Library to Optimize AUC (AUROC, AUPRC).

Logo by Zhuoning Yuan LibAUC: A Machine Learning Library for AUC Optimization Website | Updates | Installation | Tutorial | Research | Github LibAUC a

Optimization for AI 176 Jan 07, 2023
This is the official implement of paper "ActionCLIP: A New Paradigm for Action Recognition"

This is an official pytorch implementation of ActionCLIP: A New Paradigm for Video Action Recognition [arXiv] Overview Content Prerequisites Data Prep

268 Jan 09, 2023
Process text, including tokenizing and representing sentences as vectors and Applying some concepts like RNN, LSTM and GRU to create a classifier can detect the language in which a sentence is written from among 17 languages.

Language Identifier What is this ? The goal of this project is to create a model that is able to predict a given sentence language through text proces

Hossam Asaad 9 Dec 15, 2022
DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs

DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs Abstract: Image-to-image translation has recently achieved re

yaxingwang 23 Apr 14, 2022
Objax Apache-2Objax (🥉19 · ⭐ 580) - Objax is a machine learning framework that provides an Object.. Apache-2 jax

Objax Tutorials | Install | Documentation | Philosophy This is not an officially supported Google product. Objax is an open source machine learning fr

Google 729 Jan 02, 2023