Kaggle-titanic - A tutorial for Kaggle's Titanic: Machine Learning from Disaster competition. Demonstrates basic data munging, analysis, and visualization techniques. Shows examples of supervised machine learning techniques.

Overview

Kaggle-titanic

This is a tutorial in an IPython Notebook for the Kaggle competition, Titanic Machine Learning From Disaster. The goal of this repository is to provide an example of a competitive analysis for those interested in getting into the field of data analytics or using python for Kaggle's Data Science competitions .

Quick Start: View a static version of the notebook in the comfort of your own web browser.

Installation:

To run this notebook interactively:

  1. Download this repository in a zip file by clicking on this link or execute this from the terminal: git clone https://github.com/agconti/kaggle-titanic.git
  2. Install virtualenv.
  3. Navigate to the directory where you unzipped or cloned the repo and create a virtual environment with virtualenv env.
  4. Activate the environment with source env/bin/activate
  5. Install the required dependencies with pip install -r requirements.txt.
  6. Execute ipython notebook from the command line or terminal.
  7. Click on Titanic.ipynb on the IPython Notebook dasboard and enjoy!
  8. When you're done deactivate the virtual environment with deactivate.

Dependencies:

Kaggle Competition | Titanic Machine Learning from Disaster

The sinking of the RMS Titanic is one of the most infamous shipwrecks in history. On April 15, 1912, during her maiden voyage, the Titanic sank after colliding with an iceberg, killing 1502 out of 2224 passengers and crew. This sensational tragedy shocked the international community and led to better safety regulations for ships.

One of the reasons that the shipwreck led to such loss of life was that there were not enough lifeboats for the passengers and crew. Although there was some element of luck involved in surviving the sinking, some groups of people were more likely to survive than others, such as women, children, and the upper-class.

In this contest, we ask you to complete the analysis of what sorts of people were likely to survive. In particular, we ask you to apply the tools of machine learning to predict which passengers survived the tragedy.

This Kaggle Getting Started Competition provides an ideal starting place for people who may not have a lot of experience in data science and machine learning."

From the competition homepage.

Goal for this Notebook:

Show a simple example of an analysis of the Titanic disaster in Python using a full complement of PyData utilities. This is aimed for those looking to get into the field or those who are already in the field and looking to see an example of an analysis done with Python.

This Notebook will show basic examples of:

Data Handling

  • Importing Data with Pandas
  • Cleaning Data
  • Exploring Data through Visualizations with Matplotlib

Data Analysis

  • Supervised Machine learning Techniques: + Logit Regression Model + Plotting results + Support Vector Machine (SVM) using 3 kernels + Basic Random Forest + Plotting results

Valuation of the Analysis

  • K-folds cross validation to valuate results locally
  • Output the results from the IPython Notebook to Kaggle

Benchmark Scripts

To find the basic scripts for the competition benchmarks look in the "Python Examples" folder. These scripts are based on the originals provided by Astro Dave but have been reworked so that they are easier to understand for new comers.

Competition Website: http://www.kaggle.com/c/titanic-gettingStarted

Comments
  • output file

    output file "data/output/logitregres.csv" contains the survived values other than {0,1}

    Thanks for the great article and code. I see that direct submission of output file to kaggle results in error and it says Survived column values must be either 0 or 1.

    Am I missing something? Should I have a cutoff and turn them in to 0 or 1?

    opened by srini09 2
  • Fixed issue with bar chart

    Fixed issue with bar chart

    If auto-sorting is on (as per default), the returned series object is sorted by values, i.e. for „male“ the not-survived category is reported first and for „female“ the survived. When summing over male and female, the categories get mixed up.

    opened by metatier 2
  • Adds the updated csv files with capitalied column names. Fixed the iPyth...

    Adds the updated csv files with capitalied column names. Fixed the iPyth...

    ...on Notebook so it works with capitalized column headers. Updated the data folder with the two new csv files (train and test) as well as output/logitregres.csv.

    opened by thearpitgupta 2
  • Column headers are now capitalized

    Column headers are now capitalized

    Looks like column headers in the training data set are now capitalized. See here http://www.kaggle.com/c/titanic-gettingStarted/download/train.csv It's not capitalized in the data set used used in the repo https://github.com/agconti/kaggle-titanic/blob/master/data/train.csv Wonder if Kaggle changed the data set and intentionally made this change. Anyways, if you want I am happy to submit a PR that works with capitalized column names. Let me know. Thanks.

    PS - Great work.

    opened by thearpitgupta 2
  • sharey for subplots

    sharey for subplots

    Not sure if you're original intention was to show the Y axis for all your subplots in input 14 but if it wasn't you can pass in sharey=True into df.plot() function to eliminate the redundant axes.

    example

    Awesome work on the notebook btw!

    opened by zunayed 2
  • Install KaggleAux through pip

    Install KaggleAux through pip

    Currently, a subsection of KaggleAux is included in this repository as a temporary connivence. It would be cleaner to have KaggleAux as a 3rd party dependency installed through pip. This would be less confusing to users, and would allow updates in KaggleAux to be better incorporated.

    enhancement 
    opened by agconti 1
  • Categorization of algorithms

    Categorization of algorithms

    The README and several places in notebook categorize SVM and Random Forest into "Unsupervised Learning". They actually belong to "Supervised Learning".

    e.g. http://cs229.stanford.edu/notes/cs229-notes3.pdf

    opened by hupili 1
  • Suggestion -- update requirements.txt

    Suggestion -- update requirements.txt

    Hi, I don't know if this repo is still maintained, but would be nice to update the requirements.txt with supported versions.

    :+1: Thanks for putting this repo together.

    opened by DaveOkpare 0
  • Update agc_simp_gendermodel.py

    Update agc_simp_gendermodel.py

    data indexing was inappropriate for the operation taking care Lines #18,#19 index 3 , we have Name of passenger but not gender , so all the time we'll get false
    Lines #26,#27,#28 , proportions should be calculated on Survived column , not on PassengerId

    opened by praveenbommali 0
  • why use barh and ylim

    why use barh and ylim

    I don't understand the need of using barh and ylim functions in plotting.Simple vertical graphs are easier to understand then what is the purpose of using barh.And Thank you for sharing this notebook it's really informative.

    opened by barotdhrumil21 0
Releases(v0.2.0)
PlenOctree Extraction algorithm

PlenOctrees_NeRF-SH This is an implementation of the Paper PlenOctrees for Real-time Rendering of Neural Radiance Fields. Not only the code provides t

49 Nov 05, 2022
Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces

This repository contains source code for the paper Combining Latent Space and Structured Kernels for Bayesian Optimization over Combinatorial Spaces a

9 Nov 21, 2022
Official repository of Semantic Image Matting

Semantic Image Matting This is the official repository of Semantic Image Matting (CVPR2021). Overview Natural image matting separates the foreground f

192 Dec 29, 2022
Database Reasoning Over Text project for ACL paper

Database Reasoning over Text This repository contains the code for the Database Reasoning Over Text paper, to appear at ACL2021. Work is performed in

Facebook Research 320 Dec 12, 2022
HAR-stacked-residual-bidir-LSTMs - Deep stacked residual bidirectional LSTMs for HAR

HAR-stacked-residual-bidir-LSTM The project is based on this repository which is presented as a tutorial. It consists of Human Activity Recognition (H

Guillaume Chevalier 287 Dec 27, 2022
GANimation: Anatomically-aware Facial Animation from a Single Image (ECCV'18 Oral) [PyTorch]

GANimation: Anatomically-aware Facial Animation from a Single Image [Project] [Paper] Official implementation of GANimation. In this work we introduce

Albert Pumarola 1.8k Dec 28, 2022
Speech Recognition using DeepSpeech2.

deepspeech.pytorch Implementation of DeepSpeech2 for PyTorch using PyTorch Lightning. The repo supports training/testing and inference using the DeepS

Sean Naren 2k Jan 04, 2023
Codes for AAAI22 paper "Learning to Solve Travelling Salesman Problem with Hardness-Adaptive Curriculum"

Paper For more details, please see our paper Learning to Solve Travelling Salesman Problem with Hardness-Adaptive Curriculum which has been accepted a

14 Sep 30, 2022
Unifying Global-Local Representations in Salient Object Detection with Transformer

GLSTR (Global-Local Saliency Transformer) This is the official implementation of paper "Unifying Global-Local Representations in Salient Object Detect

11 Aug 24, 2022
Spectral normalization (SN) is a widely-used technique for improving the stability and sample quality of Generative Adversarial Networks (GANs)

Why Spectral Normalization Stabilizes GANs: Analysis and Improvements [paper (NeurIPS 2021)] [paper (arXiv)] [code] Authors: Zinan Lin, Vyas Sekar, Gi

Zinan Lin 32 Dec 16, 2022
A CNN model to detect hand gestures.

Software Used python - programming language used, tested on v3.8 miniconda - for managing virtual environment Libraries Used opencv - pip install open

Shivanshu 6 Jul 14, 2022
Bayesian Neural Networks in PyTorch

We present the new scheme to compute Monte Carlo estimator in Bayesian VI settings with almost no memory cost in GPU, regardles of the number of sampl

Jurijs Nazarovs 7 May 03, 2022
Transformer Tracking (CVPR2021)

TransT - Transformer Tracking [CVPR2021] Official implementation of the TransT (CVPR2021) , including training code and trained models. We are revisin

chenxin 465 Jan 06, 2023
Deep learning with dynamic computation graphs in TensorFlow

TensorFlow Fold TensorFlow Fold is a library for creating TensorFlow models that consume structured data, where the structure of the computation graph

1.8k Dec 28, 2022
A semismooth Newton method for elliptic PDE-constrained optimization

sNewton4PDEOpt The Python module implements a semismooth Newton method for solving finite-element discretizations of the strongly convex, linear ellip

2 Dec 08, 2022
DLL: Direct Lidar Localization

DLL: Direct Lidar Localization Summary This package presents DLL, a direct map-based localization technique using 3D LIDAR for its application to aeri

Service Robotics Lab 127 Dec 16, 2022
ANEA: Automated (Named) Entity Annotation for German Domain-Specific Texts

ANEA The goal of Automatic (Named) Entity Annotation is to create a small annotated dataset for NER extracted from German domain-specific texts. Insta

Anastasia Zhukova 2 Oct 07, 2022
Identify the emotion of multiple speakers in an Audio Segment

MevonAI - Speech Emotion Recognition Identify the emotion of multiple speakers in a Audio Segment Report Bug · Request Feature Try the Demo Here Table

Suyash More 110 Dec 03, 2022
Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model

Doubly Robust Off-Policy Evaluation for Ranking Policies under the Cascade Behavior Model About This repository contains the code to replicate the syn

Haruka Kiyohara 12 Dec 07, 2022
Learning to Estimate Hidden Motions with Global Motion Aggregation

Learning to Estimate Hidden Motions with Global Motion Aggregation (GMA) This repository contains the source code for our paper: Learning to Estimate

Shihao Jiang (Zac) 221 Dec 18, 2022