Dcf-game-infrastructure-public - Contains all the components necessary to run a DC finals (attack-defense CTF) game from OOO

Overview

dcf-game-infrastructure

All the components necessary to run a game of the OOO DC CTF finals.

Authors: adamd, hacopo, Erik Trickel, Zardus, and bboe

Design Philosophy

This repo contains all the game components necessary to run an Attack-Defense CTF that OOO used from 2018--2021.

The design is based on adamd's experience building the ictf-framework.

There are fundamental tenenats that we try to follow in the design of the system:

Spoke component model

The communication design of the components in the system (which you can kind of think of as micro-services) is a "spoke" model, where every component talks to the database (through a RESTish API), and no component directly talks to any other.

In this way, each component can be updated separately and can also be scaled independently using our k8s hosting.

This also made testing of each component easier, as the only dependence on a component is on the state of the database.

The only exception to this is the patchbot (the component that needs to test the patches submitted by the teams).

The database API puts the patchbot testing jobs into an RQ (Redis Queue), which all the patchbot workers pull jobs from.

Append-only database design

Fundamentally, a CTF database needs to calculate scores (that's essentially what the teams care about).

Prior design approaches that we've used would have a points or score column in the team table, and when they acquired or lost points, the app code would change this value.

However, many crazy things can happen during a CTF: recalculating scores or missed flags, even changing the scoring functions itself.

These can be difficult to handle depending on how the system is developed.

Therefore, we created a completely append-only database model, where no data in the DB is ever deleted or changed.

Even things like service status (the GOOD, OK, LOW, BAD that we used) is not a column in the services table. Every change of status would created a new StatusIndicator row, and the services would pull the latest version from this table.

Event model

Related to the append-only database design, everything in the database was represented by events.

The database would store all game events (in our game over the years was SLA_SCRIPT, FLAG_STOLEN, SET_FLAG, KOH_SCORE_FETCH, KOH_RANKING, PCAP_CREATED, PCAP_RELEASED, and STEALTH).

Then, the state of the game is based on these events.

An additional benefit is that these events could be shipped to the teams as part of the game_state.json.

Separate k8s clusters

How we ran this is with two k8s clusters: an admin cluster and a game cluster.

The admin cluster ran all of these components.

The game cluster ran all of the CTF challenges.

We used this design to do things like drop flags on the services. The flagbot used kubectl to drop a flag onto a service running in the other cluster.

This also allowed us to lock down the game cluster so that the vulnerable services couldn't make external requests, could be scaled separately, etc.

Install Requirements

This package is pip installable, and installs all dependencies. Do the following in a virtualenv:

$ pip install -e .

NOTE: If you want to connect to a mysql server (such as in prod or when deving against a mysql server), install the mysqlclient dependency like so:

$ pip install -e .[mysql]

Testing

Make sure the tests pass before you commit, and add new test cases in test for new features.

Note the database API now checks that the timezone is in UTC, so you'll need to specify that to run the tests:

$ TZ=UTC nosetests -v

Local Dev

If you're using tmux, I created a script local_dev.sh that will run a database-api, database-api frontend, team-interface backend, team-interface frontend, gamebot, and an ipython session with a database client created.

Just run the following

$ ./local_dev.sh

Deploy to prod

Build and -p push the image to production registry.

$ ./deploy.sh -p

Won't -r restart the running services, need to do:

$ ./deploy.sh -p -r

database-api

This has the tables for the database, a REST API to access it, and a python client to access the REST API.

See ooogame/database for details.

flagbot

Responsible for putting new flags into all the services for every game tick.

See ooogame/flagbot for details.

fresh-flagbot

Responsible for putting a new flags into a pod when it first comes up (from a team patching the service).

See ooogame/fresh_flagbot for details.

gamebot

Responsible for incrementing the game's ticks.

See ooogame/gamebot for details.

koh-scorebot

Responsible for extracting the King of the Hill (koh) scores from all the koh pods every tick, and submitting them to the database.

See ooogame/koh_scorebot for details.

team-interface

Responsible for providing an interface to the teams so that they can submit flags, get pcaps, upload patches, and get their patch status. Split into a backend flask REST API, which essentially wraps the database-api, and a React frontend.

See ooogame/team_interface for details.

pcapbot

Responsible for picking up all the newly generated pcaps, anonymize them, and if the service is releasing pcaps then release them.

See ooogame/pcapbot for details.

gamestatebot

Responsible for creating the game state at every new tick and storing them in the nfs, and release them publicly.

See ooogame/gamestatebot for details.

This is also the component that pushes data to the public scoreboard

Owner
Order of the Overflow
Order of the Overflow
Blender scripts for computing geodesic distance

GeoDoodle Geodesic distance computation for Blender meshes Table of Contents Overivew Usage Implementation Overview This addon provides an operator fo

20 Jun 08, 2022
Explaining in Style: Training a GAN to explain a classifier in StyleSpace

Explaining in Style: Official TensorFlow Colab Explaining in Style: Training a GAN to explain a classifier in StyleSpace Oran Lang, Yossi Gandelsman,

Google 197 Nov 08, 2022
使用yolov5训练自己数据集(详细过程)并通过flask部署

使用yolov5训练自己的数据集(详细过程)并通过flask部署 依赖库 torch torchvision numpy opencv-python lxml tqdm flask pillow tensorboard matplotlib pycocotools Windows,请使用 pycoc

HB.com 19 Dec 28, 2022
Vehicle Detection Using Deep Learning and YOLO Algorithm

VehicleDetection Vehicle Detection Using Deep Learning and YOLO Algorithm Dataset take or find vehicle images for create a special dataset for fine-tu

Maryam Boneh 96 Jan 05, 2023
MIMO-UNet - Official Pytorch Implementation

MIMO-UNet - Official Pytorch Implementation This repository provides the official PyTorch implementation of the following paper: Rethinking Coarse-to-

Sungjin Cho 248 Jan 02, 2023
Implements a fake news detection program using classifiers.

Fake news detection Implements a fake news detection program using classifiers for Data Mining course at UoA. Description The project is the categoriz

Apostolos Karvelas 1 Jan 09, 2022
A repository built on the Flow software package to explore cyber-security attacks on intelligent transportation systems.

A repository built on the Flow software package to explore cyber-security attacks on intelligent transportation systems.

George Gunter 4 Nov 14, 2022
A 1.3B text-to-image generation model trained on 14 million image-text pairs

minDALL-E on Conceptual Captions minDALL-E, named after minGPT, is a 1.3B text-to-image generation model trained on 14 million image-text pairs for no

Kakao Brain 604 Dec 14, 2022
Deep Image Search is an AI-based image search engine that includes deep transfor learning features Extraction and tree-based vectorized search.

Deep Image Search - AI-Based Image Search Engine Deep Image Search is an AI-based image search engine that includes deep transfer learning features Ex

139 Jan 01, 2023
Styleformer - Official Pytorch Implementation

Styleformer -- Official PyTorch implementation Styleformer: Transformer based Generative Adversarial Networks with Style Vector(https://arxiv.org/abs/

Jeeseung Park 159 Dec 12, 2022
M2MRF: Many-to-Many Reassembly of Features for Tiny Lesion Segmentation in Fundus Images

M2MRF: Many-to-Many Reassembly of Features for Tiny Lesion Segmentation in Fundus Images This repo is the official implementation of paper "M2MRF: Man

12 Dec 14, 2022
PyTorch implementation for View-Guided Point Cloud Completion

PyTorch implementation for View-Guided Point Cloud Completion

22 Jan 04, 2023
Out-of-distribution detection using the pNML regret. NeurIPS2021

OOD Detection Load conda environment conda env create -f environment.yml or install requirements: while read requirement; do conda install --yes $requ

Koby Bibas 23 Dec 02, 2022
HAT: Hierarchical Aggregation Transformers for Person Re-identification

HAT: Hierarchical Aggregation Transformers for Person Re-identification

11 Sep 05, 2022
Generating Images with Recurrent Adversarial Networks

Generating Images with Recurrent Adversarial Networks Python (Theano) implementation of Generating Images with Recurrent Adversarial Networks code pro

Daniel Jiwoong Im 121 Sep 08, 2022
Official implementation for "QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation" (CVPR 2022)

QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation (CVPR2022) https://arxiv.org/abs/2203.08483 Unpaired image-to-image (I2I

Xueqi Hu 50 Dec 16, 2022
A light weight data augmentation tool for training CNNs and Viola Jones detectors

hey-daug A light weight data augmentation tool for training CNNs and Viola Jones detectors (Haar Cascades). This tool inflates your data by up to six

Jaiyam Sharma 2 Nov 23, 2019
FluxTraining.jl gives you an endlessly extensible training loop for deep learning

A flexible neural net training library inspired by fast.ai

86 Dec 31, 2022
Multilingual Image Captioning

Multilingual Image Captioning Authors: Bhavitvya Malik, Gunjan Chhablani Demo Link: https://huggingface.co/spaces/flax-community/multilingual-image-ca

Gunjan Chhablani 32 Nov 25, 2022
StrongSORT: Make DeepSORT Great Again

StrongSORT StrongSORT: Make DeepSORT Great Again StrongSORT: Make DeepSORT Great Again Yunhao Du, Yang Song, Bo Yang, Yanyun Zhao arxiv 2202.13514 Abs

369 Jan 04, 2023