Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Last update: Dec 27, 2022

Related tags

Deep Learning SW-CV-ModelZoo

Overview

SW-CV-ModelZoo

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Framework: TF/Keras 2.7

Training SQLite DB built using fire-egg's tools: https://github.com/fire-eggs/Danbooru2019

Currently training on Danbooru2021, 512px SFW subset (sans the rating:q images that had been included in the 2022-01-21 release of the dataset)

Reference:

Anonymous, The Danbooru Community, & Gwern Branwen; “Danbooru2021: A Large-Scale Crowdsourced and Tagged Anime Illustration Dataset”, 2022-01-21. Web. Accessed 2022-01-28 https://www.gwern.net/Danbooru2021

Journal

06/02/2022: great news crew! TRC allowed me to use a bunch of TPUs!

To make better use of this amount of compute I had to overhaul a number of components, so a bunch of things are likely to have fallen to bitrot in the process. I can only guarantee NFNet can work pretty much as before with the right arguments.
NFResNet changes should have left it retrocompatible with the previous version.
ResNet has been streamlined to be mostly in line with the Bag-of-Tricks paper (arXiv:1812.01187) with the exception of the stem. It is not compatible with the previous version of the code.

The training labels have been included in the 2021_0000_0899 folder for convenience.
The list of files used for training is going to be uploaded as a GitHub Release.

Now for some numbers:
compared to my previous best run, the one that resulted in NFNetL1V1-100-0.57141:

I'm using 1.86x the amount of images: 2.8M vs 1.5M
I'm training bigger models: 61M vs 45M params
... in less time: 232 vs 700 hours of processor time
don't get me started on actual wall clock time
with a few amenities thrown in: ECA for channel attention, SiLU activation

And it's all thanks to the folks at TRC, so shout out to them!

I currently have a few runs in progress across a couple of dimensions:

effect of model size with NFNet L0/L1/L2, with SiLU and ECA for all three of them
effect of activation function with NFNet L0, with SiLU/HSwish/ReLU, no ECA

Once the experiments are over, the plan is to select the network definitions that lay on the Pareto curve between throughput and F1 score and release the trained weights.

One last thing.
I'd like to call your attention to the tools/cleanlab_stuff.py script.
It reads two files: one with the binarized labels from the database, the other with the predicted probabilities.
It then uses the cleanlab package to estimate whether if an image in a set could be missing a given label. At the end it stores its conclusions in a json file.
This file could, potentially, be used in some tool to assist human intervention to add the missing tags.

You might also like...

Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

71 Jan 5, 2023

Graph Neural Networks with Keras and Tensorflow 2.

Welcome to Spektral Spektral is a Python library for graph deep learning, based on the Keras API and TensorFlow 2. The main goal of this project is to

2.2k Jan 8, 2023

QKeras: a quantization deep learning library for Tensorflow Keras

QKeras github.com/google/qkeras QKeras 0.8 highlights: Automatic quantization using QKeras; Stochastic behavior (including stochastic rouding) is disa

437 Jan 3, 2023

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

Hyperparameter Optimization for Keras Talos • Key Features • Examples • Install • Support • Docs • Issues • License • Download Talos radically changes

1.6k Dec 15, 2022

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

5.7k Jan 9, 2023

Deep GPs built on top of TensorFlow/Keras and GPflow

GPflux Documentation | Tutorials | API reference | Slack What does GPflux do? GPflux is a toolbox dedicated to Deep Gaussian processes (DGP), the hier

107 Nov 2, 2022

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

tf2onnx converts TensorFlow (tf-1.x or tf-2.x), tf.keras and tflite models to ONNX via command line or python api.

1.8k Jan 8, 2023

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

deep_autoviml Build keras pipelines and models in a single line of code! Table of Contents Motivation How it works Technology Install Usage API Image

102 Dec 17, 2022

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

1.5k Jan 3, 2023

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

ConvNext B, ViT B16
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
top 5500 tags (2021_0000_0899_5500/selected_tags.csv)
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_09_25_2022_05h13m55s | B | 93.2M | 448 | 0.3673 | 0.6941 | | ViTB16_09_25_2022_04h53m38s | B16 | 90.5M | 448 | 0.3663 | 0.6918 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_09_25_2022_05h13m55s.7z(322.58 MB)
ViTB16_09_25_2022_04h53m38s.7z(312.96 MB)
convnexts_db2021_2022_03_22(Mar 22, 2022)

ConvNext, T/S/B
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_03_10_2022_21h41m23s | B | 90.01M | 448 | 0.3372 | 0.6892 | | ConvNextSV1_03_11_2022_17h49m56s | S | 51.28M | 384 | 0.3301 | 0.6798 | | ConvNextTV1_03_05_2022_15h56m42s | T | 29.65M | 320 | 0.3259 | 0.6595 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_03_10_2022_21h41m23s.7z(311.29 MB)
ConvNextSV1_03_11_2022_17h49m56s.7z(177.36 MB)
ConvNextTV1_03_05_2022_15h56m42s.7z(102.96 MB)
nfnets_db2021_2022_03_04(Mar 4, 2022)

NFNet, L0/L1/L2 (based on timm Lx model definitions) Trained on Danbooru2021 512px SFW subset, modulos 0000-0899 alpha to white padding to make the image square is white channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | NFNetL2V1_02_20_2022_10h27m08s | L2 | 60.96M | 448 | 0.3231 | 0.6785 | | NFNetL1V1_02_17_2022_20h18m38s | L1 | 45.65M | 384 | 0.3259 | 0.6691 | | NFNetL0V1_02_10_2022_17h50m14s | L0 | 27.32M | 320 | 0.3190 | 0.6509 |
Source code(tar.gz)
Source code(zip)
NFNetL0V1_02_10_2022_17h50m14s.7z(94.98 MB)
NFNetL1V1_02_17_2022_20h18m38s.7z(157.97 MB)
NFNetL2V1_02_20_2022_10h27m08s.7z(210.49 MB)
nfnet_tpu_training(Feb 6, 2022)

Attaching the lists of files currently in use for training and validation, along with the selected labels.

Training and validation files have been selected from the 512px SFW subset.
Source code(tar.gz)
Source code(zip)
2021_0000_0899.7z(1.32 MB)
NFNetL1V1-100-0.57141(Dec 31, 2021)
NFNet, L1 (based on timm Lx model definitions), 100 epochs, F1 @ 0.4 at the end of the 100th epoch was 0.57141

Trained on Danbooru2020 512px SFW subset, modulos 0000-0599

320px per side

alpha to white

padding to make the image square is white

channel order is BGR, scaled to 0-1

mixup alpha = 0.2 during epochs 76-100

analyze_metrics on Danbooru2020 original set, modulos 0984-0999: {'thres': 0.3485, 'F1': 0.6133, 'F2': 0.6133, 'MCC': 0.6094, 'A': 0.9923, 'R': 0.6133, 'P': 0.6133}

analyze_metrics on image IDs 4970000-5000000: {'thres': 0.3148, 'F1': 0.5942, 'F2': 0.5941, 'MCC': 0.5892, 'A': 0.9901, 'R': 0.5940, 'P': 0.5943}

Source code(tar.gz)
Source code(zip)
NFNetL1V1-100-0.57141.7z(158.09 MB)

Owner

GitHub Repository

⚖️🔁🔮🕵️‍♂️🦹🖼️ Code for Measuring the Contribution of Multiple Model Representations in Detecting Adversarial Instances paper.

Measuring the Contribution of Multiple Model Representations in Detecting Adversarial Instances This repository contains the code for Measuring the Co

0 Nov 06, 2022

A benchmark framework for Tensorflow

TensorFlow benchmarks This repository contains various TensorFlow benchmarks. Currently, it consists of two projects: PerfZero: A benchmark framework

1.1k Dec 30, 2022

Using Hotel Data to predict High Value And Potential VIP Guests

Description Using hotel data and AI to predict high value guests and potential VIP guests. Hotel can leverage on prediction resutls to run more effect

12 Feb 14, 2022

Official repo for SemanticGAN https://nv-tlabs.github.io/semanticGAN/

SemanticGAN This is the official code for: Semantic Segmentation with Generative Models: Semi-Supervised Learning and Strong Out-of-Domain Generalizat

151 Dec 28, 2022

v objective diffusion inference code for JAX.

v-diffusion-jax v objective diffusion inference code for JAX, by Katherine Crowson (@RiversHaveWings) and Chainbreakers AI (@jd_pressman). The models

186 Dec 21, 2022

Official implementation of "Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision" ECCV2020

XDVioDet Official implementation of "Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision" ECCV2020. The proj

64 Dec 12, 2022

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Related tags

Overview

SW-CV-ModelZoo

Reference:

Journal

You might also like...

Human head pose estimation using Keras over TensorFlow.

Graph Neural Networks with Keras and Tensorflow 2.

QKeras: a quantization deep learning library for Tensorflow Keras

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

Deep GPs built on top of TensorFlow/Keras and GPflow

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

convnexts_db2021_2022_03_22(Mar 22, 2022)

nfnets_db2021_2022_03_04(Mar 4, 2022)

nfnet_tpu_training(Feb 6, 2022)

NFNetL1V1-100-0.57141(Dec 31, 2021)

Owner

⚖️🔁🔮🕵️‍♂️🦹🖼️ Code for *Measuring the Contribution of Multiple Model Representations in Detecting Adversarial Instances* paper.

A benchmark framework for Tensorflow

Using Hotel Data to predict High Value And Potential VIP Guests

Official repo for SemanticGAN https://nv-tlabs.github.io/semanticGAN/

v objective diffusion inference code for JAX.

Official implementation of "Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision" ECCV2020

Convert Python 3 code to CUDA code.

Learning Features with Parameter-Free Layers (ICLR 2022)

Fast RFC3339 compliant Python date-time library

Neural style transfer as a class in PyTorch

[arXiv'22] Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation

[NeurIPS 2020] Blind Video Temporal Consistency via Deep Video Prior

Pytorch implementation AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networks

The fundamental package for scientific computing with Python.

Semantic Edge Detection with Diverse Deep Supervision

Based on Stockfish neural network(similar to LcZero)

Art Project "Schrödinger's Game of Life"

AITom is an open-source platform for AI driven cellular electron cryo-tomography analysis.

Python interface for SmartRF Sniffer 2 Firmware

[NeurIPS 2020] Code for the paper "Balanced Meta-Softmax for Long-Tailed Visual Recognition"

⚖️🔁🔮🕵️‍♂️🦹🖼️ Code for Measuring the Contribution of Multiple Model Representations in Detecting Adversarial Instances paper.