Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Last update: Dec 29, 2021

Overview

Constrained Logistic Regression

Sample implementation of constructing a logistic regression with given ranges on each of the feature's coefficients (via clogistic library).

The Data

We will use the processed version of telco customer churn data from Kaggle. The data can be downloaded here.

Steps

Define the constraints

For example:

# define constraints as dataframe
import numpy as np
constraint_df = pd.DataFrame(data=[
                                   ['gender',-np.inf,np.inf],
                                   ['SeniorCitizen',-np.inf,np.inf],
                                   ['Partner',-np.inf, 0],
                                   ['Dependents',-np.inf,0],
                                   ['tenure',-np.inf,0],
                                   ['PhoneService',-np.inf,0],
                                   ['PaperlessBilling',-np.inf,np.inf],
                                   ['MonthlyCharges',-np.inf,np.inf],
                                   ['intercept',-np.inf,np.inf]],
                             columns=['feature','lower_bound','upper_bound'])
constraint_df

|    | feature          |   lower_bound |   upper_bound |
|---:|:-----------------|--------------:|--------------:|
|  0 | gender           |          -inf |           inf |
|  1 | SeniorCitizen    |          -inf |           inf |
|  2 | Partner          |          -inf |             0 |
|  3 | Dependents       |          -inf |             0 |
|  4 | tenure           |          -inf |             0 |
|  5 | PhoneService     |          -inf |             0 |
|  6 | PaperlessBilling |          -inf |           inf |
|  7 | MonthlyCharges   |          -inf |           inf |
|  8 | intercept        |          -inf |           inf |

Model training via clogistic

# train using clogistic
from scipy.optimize import Bounds
from clogistic import LogisticRegression as clLogisticRegression

lower_bounds = constraint_df['lower_bound'].to_numpy()
upper_bounds = constraint_df['upper_bound'].to_numpy()
bounds = Bounds(lower_bounds, upper_bounds)

cl_logreg = clLogisticRegression(penalty='none')
cl_logreg.fit(X_train, y_train, bounds=bounds)

Retrieve the model coefficients

# coefficients as dataframe
cl_coef = pd.DataFrame({
    'feature': df.drop(columns='Churn').columns.tolist() + ['intercept'],
    'coefficient': list(cl_logreg.coef_[0]) + [cl_logreg.intercept_[0]]
})

cl_coef

|    | feature          |   coefficient |
|---:|:-----------------|--------------:|
|  0 | gender           |   0.0184168   |
|  1 | SeniorCitizen    |   0.506692    |
|  2 | Partner          |   3.85603e-09 |
|  3 | Dependents       |  -0.35721     |
|  4 | tenure           |  -0.0557211   |
|  5 | PhoneService     |  -0.796233    |
|  6 | PaperlessBilling |   0.398824    |
|  7 | MonthlyCharges   |   0.033197    |
|  8 | intercept        |  -1.36086     |

Constrained Logistic Regression - How to apply specific constraints to logistic regression's coefficients

Related tags

Overview

Constrained Logistic Regression

The Data

Steps

Define the constraints

Model training via clogistic

Owner

Official implementation of "Membership Inference Attacks Against Self-supervised Speech Models"

Official Implementation for Fast Training of Neural Lumigraph Representations using Meta Learning.

This is the first released system towards complex meters` detection and recognition, which is implemented by computer vision techniques.

Learning Intents behind Interactions with Knowledge Graph for Recommendation, WWW2021

Implementation for Shape from Polarization for Complex Scenes in the Wild

Physics-informed Neural Operator for Learning Partial Differential Equation

Do Smart Glasses Dream of Sentimental Visions? Deep Emotionship Analysis for Eyewear Devices

A Home Assistant custom component for Lobe. Lobe is an AI tool that can classify images.

Code for the paper "Improving Vision-and-Language Navigation with Image-Text Pairs from the Web" (ECCV 2020)

Face detection using deep learning.

Episodic Transformer (E.T.) is a novel attention-based architecture for vision-and-language navigation. E.T. is based on a multimodal transformer that encodes language inputs and the full episode history of visual observations and actions.

Implementation of paper "Self-supervised Learning on Graphs:Deep Insights and New Directions"

🐾 Semantic segmentation of paws from cute pet images (PyTorch)

InvTorch: memory-efficient models with invertible functions

Python lib to talk to pylontech lithium batteries (US2000, US3000, ...) using RS485

Wandb-predictions - WANDB Predictions With Python

AI Based Smart Exam Proctoring Package

End-to-End Object Detection with Fully Convolutional Network

Joint-task Self-supervised Learning for Temporal Correspondence (NeurIPS 2019)

The repository forked from NVlabs uses our data. (Differentiable rasterization applied to 3D model simplification tasks)