Credit Card Fraud Detection

Overview

Credit Card Fraud Detection

image


For this project, I used the datasets from the kaggle competition called IEEE-CIS Fraud Detection. The competition aims to improve fraud prevention system by building fraud detection models based on Vesta Corporation's real-world e-commerce transactional data, which contains information from device type to product features. My personal goal for this project is to not only explore the data and build models, but to also build an API server with retrainable model. To achieve this goal, I used fastapi, lightgbm and ray tune.

Also, I decided to develop this project to be the same as how data-related projects are developed in real-world scenarios, wherein the end goal of development is a project that is feasible for production. Therefore, I have put efforts on creating:

  1. Exploratory Data Analysis (EDA) in the notebooks/ folder;
  2. An API Server inside the api/ folder;
  3. Files for deployment such as Dockerfile and docker-compose.yml;
  4. Documentations in the docs/ folder; and
  5. Some necessary scripts in scripts/ folder.

Services

I have two services: app and script.

App

image The app service is a machine learning API that is open on port 8000. I used fastapi for the API server, so you can check it on http://localhost:8000/docs after you run the app service.

Scripts

image The script sends the request for the predictions on new sets of data, such as the kaggle testing data. After the script get all the responses, files will be written on /tmp/submission.csv (on host and container), but this part can take a lot of time. It is suggested to use docker logs -f lightgbm-project-demo_script_1 to check the progress of the process.

How to run this demo

1. Install the requirements

  • docker
  • docker-compose
  • make

2. Download the datasets

Option 1.

Setup kaggle API and use

make init-data

Option 2.

  1. Create a data folder: i.e.
mkdir data
  1. Download the data from kaggle IEEE-CIS Fraud Detection.

  2. Put the ieee-fraud-detection.zip inside the data/ folder.

  3. Unzip ieee-fraud-detection.zip.

3. Build the image

make build

4. Start the services

Start both the two services

docker-compose up

or only start the app service using

docker-compose up app

image

Here are some documentations

How to set up the working environment for this project

API example

Note

If you want to change the hyperparameters search space, you can go to config.py. Or you even want to use other framework the build the model, I think this demo is detail enough as a reference for your project.

About the hyperparameter search algorithm, I am using the random search for this demo but if you want to try other searching algorithm, you can change train.py.

I have another project use Tensorflow as the back bone model. Take a look about the project lightgbm-project-demo

Owner
RayWu
RayWu
Platform Tree for Xiaomi Redmi Note 7/7S (lavender)

The Xiaomi Redmi Note 7 (codenamed "lavender") is a mid-range smartphone from Xiaomi announced in January 2019. Device specifications Device Xiaomi Re

MUHAMAD KHOIRON 2 Dec 20, 2021
Multitrack exporter for OP-Z

Underbridge for OP-Z Multitrack exporter Description Exports patterns and projects individual audio tracks to seperate folders for use in your DAW. Py

Thomas Herrmann 71 Dec 25, 2022
Extra scripts to improve user experience related to OpenTaiko

OpenTaiko-Utils Extra scripts to improve user experience related to OpenTaiko osu2tja /!\ IMPORTANT NOTE /!\ Converted charts that aren't yours are fo

2 Dec 25, 2022
Flask-built web application that simulates a time and cost calculator for charging Electric Vehicles.

ev_charging_calculator Flask-built web application that simulates a time and cost calculator for charging Electric Vehicles. The project aims to simul

1 Nov 03, 2021
Eros is an expiremental programming language built using simple Python code.

Eros is an expiremental programming language built using simple Python code. Featuring an easy syntax and unique features like type slicing, the language remains an expirement that grows in down time

zxro 2 Nov 21, 2021
Python implementation of the Lox language from Robert Nystrom's Crafting Interpreters

pylox Python implementation of the Lox language from Robert Nystrom's Crafting Interpreters. https://craftinginterpreters.com. This only implements th

David Beazley 37 Dec 28, 2022
The official FOSSCOMM 2021 CTF by [email protected]

FOSSCOMM 2021 CTF Table of Contents General Info FAQ General Info Purpose: This CTF is a collaboration between the FOSSCOMM conference and the Machina 2 Nov 14, 2021

A companion web application to connect stash to deovr

stash-vr-companion This is a companion web application to connect stash to deovr. Stash is a self hosted web application to manage your porn collectio

19 Sep 29, 2022
Python Freecell Solver

freecell Python Freecell Solver Very early version right now. You can pick a board by changing the file path in freecell.py If you want to play a game

Ben Kaufman 1 Nov 26, 2021
Find Transposon Element insertions using long reads (nanopore), by alignment directly. (minimap2)

find_te_ins find_te_ins is designed to find Transposon Element (TE) insertions using long reads (nanopore), by alignment directly. (minimap2) Install

Ming Wang 1 Feb 09, 2022
the classic version Of torrentleechx #Unmaintained #Archived

TorrentleechX-Classic Old Modified Version Repo #Unmaintained #Archived for support join here working example group Leech Here For Any Issues/Imroveme

XcodersHub 18 Jan 30, 2022
Python package for handling and analyzing PSRFITS files

PyPulse A pure-Python package for handling and analyzing PSRFITS files. Read the documentation here. This is an alternate code base from PSRCHIVE. Req

Michael Lam 15 Nov 30, 2022
:art: Diagram as Code for prototyping cloud system architectures

Diagrams Diagram as Code. Diagrams lets you draw the cloud system architecture in Python code. It was born for prototyping a new system architecture d

MinJae Kwon 27.5k Jan 04, 2023
Student Result Management System Project in tkinter created based on python, tkinter, and SQLITE3 Database

Student-Result-Management-System This Student Result Management System Project in tkinter created based on python, tkinter, and SQLITE3 Database. The

Ravi Chauhan 2 Aug 03, 2022
An app to automatically take attendance by scanning students' bar coded ID card as they enter the classroom.

Auto Classroom Attendance This application may be run on a PC to automatically scan students' ID card using a generic bar code scanner and output the

1 Nov 10, 2021
A simply program to find active jackbox.tv game codes

PeepingJack A simply program to find active jackbox.tv game codes How does this work? It uses a threadpool to loop through all possible codes in a ran

3 Mar 20, 2022
Generating rent availability info from Effort rent

Rent-info Generating rent availability info from Effort rent Pre-Installation Latest version of python Pip module json, os, requests, datetime, time i

Laixuan 1 Oct 20, 2021
Data wrangling & common calculations for results from qMem measurement software

qMem Datawrangler This script processes output of qMem measurement software into an Origin ® compatible *.csv files and matplotlib graphs to quickly v

Julian 1 Nov 30, 2021
A code to clean and extract a bib file based on keywords.

These are two scripts I use to generate clean bib files. clean_bibfile.py: Removes superfluous fields (which are not included in fields_to_keep.json)

Antoine Allard 4 May 16, 2022
Python library for Minitel communication through serial port

Python library for Minitel communication through serial port

Noël 3 Aug 14, 2022