A PyTorch Lightning Callback for pushing models to the Hugging Face Hub 🤗⚡️

Last update: Dec 14, 2022

Overview

hf-hub-lightning

A callback for pushing lightning models to the Hugging Face Hub.

Note: I made this package for myself, mostly...if folks seem to be interested in it, we'll move this into huggingface_hub or an official PyTorch Lightning repo directly and archive this one. Since there are a lot of options/considerations, I decided to just post it separately for now as a proof of concept.

Setup

pip install hf-hub-lightning

Usage

To periodically upload your model ckpt and TensorBoard logs while training, you can do the following...

import pytorch_lightning as pl
from hf_hub_lightning import HuggingFaceHubCallback

train_dataloader = ...
model = YourCoolModel()
trainer = pl.Trainer(callbacks=[HuggingFaceHubCallback('your_username/model_id')])
trainer.fit(model, train_dataloader)

To load your model back from the huggingface hub, just do...

from huggingface_hub import hf_hub_download

ckpt_path = hf_hub_download('your_username/model_id', 'lit_model.ckpt')
model = YourCoolModel.load_from_checkpoint(ckpt_path)

You might also like...

Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

HLA-Face: Joint High-Low Adaptation for Low Light Face Detection The official PyTorch implementation for HLA-Face: Joint High-Low Adaptation for Low L

77 Dec 8, 2022

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

DVG-Face: Dual Variational Generation for HFR This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Re

52 Dec 30, 2022

[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

SADRNet Paper link: SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction Requirements python

Multimedia Computing Group, Nanjing University

99 Dec 30, 2022

Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Realtime Face Anti-Spoofing Detection 🤖 Realtime Face Anti Spoofing Detection with Face Detector to detect real and fake faces Please star this repo

86 Aug 3, 2022

Swapping face using Face Mesh with TensorFlow Lite

17 Apr 26, 2022

Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels.

The Face Synthetics dataset Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels. It was introduced in ou

608 Jan 2, 2023

Face Library is an open source package for accurate and real-time face detection and recognition

Face Library Face Library is an open source package for accurate and real-time face detection and recognition. The package is built over OpenCV and us

52 Nov 9, 2022

VGGFace2-HQ - A high resolution face dataset for face editing purpose

The first open source high resolution dataset for face swapping!!! A high resolution version of VGGFace2 for academic face editing purpose

232 Dec 29, 2022

A large-scale face dataset for face parsing, recognition, generation and editing.

CelebAMask-HQ [Paper] [Demo] CelebAMask-HQ is a large-scale face image dataset that has 30,000 high-resolution face images selected from the CelebA da

1.7k Dec 26, 2022

Comments

Saving only the best models
Hi, just curious how it is interconnected with PL internal checkpointing callback, and saving only the best models according to some given monitoring criteria like max validation accuracy etc.?

# saves a file like: my/path/sample-mnist-epoch=02-val_loss=0.32.ckpt checkpoint_callback = ModelCheckpoint( monitor="val_loss", dirpath="my/path/", filename="sample-mnist-{epoch:02d}-{val_loss:.2f}", save_top_k=3, mode="min", ) trainer = Trainer(callbacks=[checkpoint_callback])

see: https://pytorch-lightning.readthedocs.io/en/latest/common/weights_loading.html

Thx :rabbit:
opened by Borda 1

Releases(v0.0.2)

v0.0.2(Oct 27, 2021)
fix logger updates so you can view updated TB logs as your model trains

fix args to the callback, adding missing ones I had hardcoded

Source code(tar.gz)
Source code(zip)
v0.0.1(Oct 27, 2021)

Initial Release!
Source code(tar.gz)
Source code(zip)

A PyTorch Lightning Callback for pushing models to the Hugging Face Hub 🤗⚡️

Related tags

Overview

hf-hub-lightning

Setup

Usage

You might also like...

Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Swapping face using Face Mesh with TensorFlow Lite

Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels.

Face Library is an open source package for accurate and real-time face detection and recognition

VGGFace2-HQ - A high resolution face dataset for face editing purpose

A large-scale face dataset for face parsing, recognition, generation and editing.

Comments

Saving only the best models

Releases(v0.0.2)

v0.0.2(Oct 27, 2021)

v0.0.1(Oct 27, 2021)

Owner

Nathan Raw

OrienMask: Real-time Instance Segmentation with Discriminative Orientation Maps

This repository allows you to anonymize sensitive information in images/videos. The solution is fully compatible with the DL-based training/inference solutions that we already published/will publish for Object Detection and Semantic Segmentation.

Code for "My(o) Armband Leaks Passwords: An EMG and IMU Based Keylogging Side-Channel Attack" paper

(ICCV 2021 Oral) Re-distributing Biased Pseudo Labels for Semi-supervised Semantic Segmentation: A Baseline Investigation.

The toolkit to generate auto labeled datasets

[ECCV2020] Content-Consistent Matching for Domain Adaptive Semantic Segmentation

This package contains deep learning models and related scripts for RoseTTAFold

Taichi Course Homework Template

Official PyTorch implementation of "Improving Face Recognition with Large AgeGaps by Learning to Distinguish Children" (BMVC 2021)

MOT-Tracking-by-Detection-Pipeline - For Tracking-by-Detection format MOT (Multi Object Tracking), is it a framework that separates Detection and Tracking processes?

Show-attend-and-tell - TensorFlow Implementation of "Show, Attend and Tell"

Customised to detect objects automatically by a given model file(onnx)

ESGD-M - A stochastic non-convex second order optimizer, suitable for training deep learning models, for PyTorch

Materials for upcoming beginner-friendly PyTorch course (work in progress).

PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short-Term Transformer for Online Action Detection".

Classify music genre from a 10 second sound stream using a Neural Network.

Official implementation for TTT++: When Does Self-supervised Test-time Training Fail or Thrive

HybVIO visual-inertial odometry and SLAM system

Knowledge Management for Humans using Machine Learning & Tags

Fast sparse deep learning on CPUs