ML Optimizers from scratch using JAX

Last update: Jul 29, 2022

Overview

ML Optimizers from scratch using JAX

Implementations of some popular optimizers from scratch for a simple model i.e., Linear Regression on a dataset of 5 features. The goal of this project was to understand how these optimizers work under the hood and try to do a toy implementation myself. I also use a bit of JAX magic to perform the differentiation of the loss function w.r.t to the weights and the bias without explicitly writing their derivatives as a separate function. This can help to generalize this notebook for other types of loss functions as well.

The optimizers I have implemented are -

Batch Gradient Descent
Batch Gradient Descent + Momentum
Nesterov Accelerated Momentum
Adagrad
RMSprop
Adam
Adamax
Nadam
Adabelief

References -

Owner

Shreyansh Singh

AI Engineer at @Mastercard. Ex-Intern at @Samsung. CS Grad, IIT (BHU) Varanasi. CTF player with @abs0lut3pwn4g3

GitHub Repository

Model factory is a ML training platform to help engineers to build ML models at scale

Model Factory Machine learning today is powering many businesses today, e.g., search engine, e-commerce, news or feed recommendation. Training high qu

16 Sep 23, 2022

Regularization and Feature Selection in Least Squares Temporal Difference Learning

Regularization and Feature Selection in Least Squares Temporal Difference Learning Description This is Python implementations of Least Angle Regressio

0 Jan 18, 2022

Dive into Machine Learning

Dive into Machine Learning Hi there! You might find this guide helpful if: You know Python or you're learning it 🐍 You're new to Machine Learning You

11.1k Jan 03, 2023

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning.

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning. DirectML provides GPU acceleration for common machine learning tasks across a broad range of supported ha

1.1k Jan 04, 2023

Machine Learning from Scratch

Machine Learning from Scratch Author: Shengxuan Wang From: Oregon State University Content: Building Machine Learning model from Scratch, without usin

0 Jul 05, 2022

nn-Meter is a novel and efficient system to accurately predict the inference latency of DNN models on diverse edge devices

A DNN inference latency prediction toolkit for accurately modeling and predicting the latency on diverse edge devices.

241 Dec 26, 2022

CVXPY is a Python-embedded modeling language for convex optimization problems.

CVXPY The CVXPY documentation is at cvxpy.org. We are building a CVXPY community on Discord. Join the conversation! For issues and long-form discussio

4.3k Jan 08, 2023

Simple Machine Learning Tool Kit

Getting started smltk (Simple Machine Learning Tool Kit) package is implemented for helping your work during data preparation testing your model The g

1 Dec 30, 2021

CinnaMon is a Python library which offers a number of tools to detect, explain, and correct data drift in a machine learning system

67 Dec 28, 2022

Python package for concise, transparent, and accurate predictive modeling

Python package for concise, transparent, and accurate predictive modeling. All sklearn-compatible and easy to use. 📚 docs • 📖 demo notebooks Modern

983 Jan 01, 2023

MiniTorch - a diy teaching library for machine learning engineers

This repo is the full student code for minitorch. It is designed as a single repo that can be completed part by part following the guide book. It uses

1.1k Jan 07, 2023

Distributed Evolutionary Algorithms in Python

DEAP DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data stru

4.9k Jan 05, 2023

Exemplary lightweight and ready-to-deploy machine learning project

6 Dec 20, 2022

My project contrasts K-Nearest Neighbors and Random Forrest Regressors on Real World data

kNN-vs-RFR My project contrasts K-Nearest Neighbors and Random Forrest Regressors on Real World data In many areas, rental bikes have been launched to

1 Oct 28, 2021

Time series forecasting with PyTorch

Our article on Towards Data Science introduces the package and provides background information. Pytorch Forecasting aims to ease state-of-the-art time

2.5k Jan 02, 2023

A statistical library designed to fill the void in Python's time series analysis capabilities, including the equivalent of R's auto.arima function.

pmdarima Pmdarima (originally pyramid-arima, for the anagram of 'py' + 'arima') is a statistical library designed to fill the void in Python's time se

1.3k Dec 22, 2022

ML Optimizers from scratch using JAX

Related tags

Overview

ML Optimizers from scratch using JAX

Owner

Shreyansh Singh

Model factory is a ML training platform to help engineers to build ML models at scale

Regularization and Feature Selection in Least Squares Temporal Difference Learning

Dive into Machine Learning

DirectML is a high-performance, hardware-accelerated DirectX 12 library for machine learning.

Machine Learning from Scratch

nn-Meter is a novel and efficient system to accurately predict the inference latency of DNN models on diverse edge devices

CVXPY is a Python-embedded modeling language for convex optimization problems.

Simple Machine Learning Tool Kit

CinnaMon is a Python library which offers a number of tools to detect, explain, and correct data drift in a machine learning system

Python package for concise, transparent, and accurate predictive modeling

MiniTorch - a diy teaching library for machine learning engineers

Distributed Evolutionary Algorithms in Python

Exemplary lightweight and ready-to-deploy machine learning project

My project contrasts K-Nearest Neighbors and Random Forrest Regressors on Real World data

Time series forecasting with PyTorch

A statistical library designed to fill the void in Python's time series analysis capabilities, including the equivalent of R's auto.arima function.

ETNA is an easy-to-use time series forecasting framework.

TIANCHI Purchase Redemption Forecast Challenge

MLR - Machine Learning Research

The Ultimate FREE Machine Learning Study Plan