Direct application of DALLE-2 to video synthesis, using factored space-time Unet and Transformers

Last update: May 15, 2022

Overview

DALLE2 Video (wip)

** only to be built after DALLE2 image is done and replicated, and the importance of the prior network is validated **

Direct application of DALLE-2 to video synthesis, using factored space-time UNet and Transformers

CLIP could have two approaches. (1) design 3D CLIP using TimesFormer for the video encoding (2) use 2D CLIP with attention pooling at the end over all image embeddings with time positional encoding

For all you academics out there emailing me, you are free to publish this idea without citing me at all. I'm just after the world of infinite machine dreams, and I don't really care how we get there.

Owner

Phil Wang

Working with Attention. It's all we need

GitHub Repository

Code repo for realtime multi-person pose estimation in CVPR'17 (Oral)

Realtime Multi-Person Pose Estimation By Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh. Introduction Code repo for winning 2016 MSCOCO Keypoints Cha

4.9k Dec 31, 2022

Real-time ground filtering algorithm of cloud points acquired using Terrestrial Laser Scanner (TLS)

This repository contains tools to simulate the ground filtering process of a registered point cloud. The repository contains two filtering methods. The first method uses a normal vector, and fit to p

5 Aug 25, 2022

Curating a dataset for bioimage transfer learning

CytoImageNet A large-scale pretraining dataset for bioimage transfer learning. Motivation In past few decades, the increase in speed of data collectio

9 Jun 20, 2022

Self-supervised Multi-modal Hybrid Fusion Network for Brain Tumor Segmentation

JBHI-Pytorch This repository contains a reference implementation of the algorithms described in our paper "Self-supervised Multi-modal Hybrid Fusion N

5 Dec 13, 2021

Official code for "Focal Self-attention for Local-Global Interactions in Vision Transformers"

Focal Transformer This is the official implementation of our Focal Transformer -- "Focal Self-attention for Local-Global Interactions in Vision Transf

486 Dec 20, 2022

CRISCE: Automatically Generating Critical Driving Scenarios From Car Accident Sketches

CRISCE: Automatically Generating Critical Driving Scenarios From Car Accident Sketches This document describes how to install and use CRISCE (CRItical

2 Feb 09, 2022

Doing fast searching of nearest neighbors in high dimensional spaces is an increasingly important problem

Benchmarking nearest neighbors Doing fast searching of nearest neighbors in high dimensional spaces is an increasingly important problem, but so far t

3.2k Jan 03, 2023

This is an official implementation for "SimMIM: A Simple Framework for Masked Image Modeling".

Project This repo has been populated by an initial template to help get you started. Please make sure to update the content to build a great experienc

674 Dec 26, 2022

A Model for Natural Language Attack on Text Classification and Inference

TextFooler A Model for Natural Language Attack on Text Classification and Inference This is the source code for the paper: Jin, Di, et al. "Is BERT Re

418 Dec 16, 2022

[Preprint] "Bag of Tricks for Training Deeper Graph Neural Networks A Comprehensive Benchmark Study" by Tianlong Chen, Kaixiong Zhou, Keyu Duan, Wenqing Zheng, Peihao Wang, Xia Hu, Zhangyang Wang

Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study Codes for [Preprint] Bag of Tricks for Training Deeper Graph

101 Dec 29, 2022

[NeurIPS 2021] Galerkin Transformer: a linear attention without softmax

[NeurIPS 2021] Galerkin Transformer: linear attention without softmax Summary A non-numerical analyst oriented explanation on Toward Data Science abou

159 Dec 20, 2022

Code for IntraQ, PyTorch implementation of our paper under review

IntraQ: Learning Synthetic Images with Intra-Class Heterogeneity for Zero-Shot Network Quantization paper Requirements Python = 3.7.10 Pytorch == 1.7

1 Nov 19, 2021

The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter

FAPIS The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter Introduction This repo is primari

8 Dec 11, 2022

Music Classification: Beyond Supervised Learning, Towards Real-world Applications

104 Dec 15, 2022

Human-Pose-and-Motion History

Human Pose and Motion Scientist Approach Eadweard Muybridge, The Galloping Horse Portfolio, 1887 Etienne-Jules Marey, Descent of Inclined Plane, Chron

47 Dec 16, 2022

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

GPT2-Pytorch with Text-Generator Better Language Models and Their Implications Our model, called GPT-2 (a successor to GPT), was trained simply to pre

775 Jan 08, 2023

A Python library for working with arbitrary-dimension hypercomplex numbers following the Cayley-Dickson construction of algebras.

Hypercomplex A Python library for working with quaternions, octonions, sedenions, and beyond following the Cayley-Dickson construction of hypercomplex

7 Nov 04, 2022

Direct application of DALLE-2 to video synthesis, using factored space-time Unet and Transformers

Related tags

Overview

DALLE2 Video (wip)

Owner

Phil Wang

Code repo for realtime multi-person pose estimation in CVPR'17 (Oral)

Real-time ground filtering algorithm of cloud points acquired using Terrestrial Laser Scanner (TLS)

Curating a dataset for bioimage transfer learning

Self-supervised Multi-modal Hybrid Fusion Network for Brain Tumor Segmentation

Official code for "Focal Self-attention for Local-Global Interactions in Vision Transformers"

CRISCE: Automatically Generating Critical Driving Scenarios From Car Accident Sketches

Doing fast searching of nearest neighbors in high dimensional spaces is an increasingly important problem

This is an official implementation for "SimMIM: A Simple Framework for Masked Image Modeling".

A Model for Natural Language Attack on Text Classification and Inference

[Preprint] "Bag of Tricks for Training Deeper Graph Neural Networks A Comprehensive Benchmark Study" by Tianlong Chen, Kaixiong Zhou, Keyu Duan, Wenqing Zheng, Peihao Wang, Xia Hu, Zhangyang Wang

[NeurIPS 2021] Galerkin Transformer: a linear attention without softmax

Code for IntraQ, PyTorch implementation of our paper under review

The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter

Music Classification: Beyond Supervised Learning, Towards Real-world Applications

Human-Pose-and-Motion History

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

A Python library for working with arbitrary-dimension hypercomplex numbers following the Cayley-Dickson construction of algebras.

A configurable, tunable, and reproducible library for CTR prediction

Demo project for real time anomaly detection using kafka and python

Gym Threat Defense

Direct application of DALLE-2 to video synthesis, using factored space-time Unet and Transformers

Related tags

Overview

DALLE2 Video (wip)

Owner

Phil Wang

Code repo for realtime multi-person pose estimation in CVPR'17 (Oral)

Real-time ground filtering algorithm of cloud points acquired using Terrestrial Laser Scanner (TLS)

Curating a dataset for bioimage transfer learning

Self-supervised Multi-modal Hybrid Fusion Network for Brain Tumor Segmentation

Official code for "Focal Self-attention for Local-Global Interactions in Vision Transformers"

CRISCE: Automatically Generating Critical Driving Scenarios From Car Accident Sketches

Doing fast searching of nearest neighbors in high dimensional spaces is an increasingly important problem

This is an official implementation for "SimMIM: A Simple Framework for Masked Image Modeling".

A Model for Natural Language Attack on Text Classification and Inference

[Preprint] "Bag of Tricks for Training Deeper Graph Neural Networks A Comprehensive Benchmark Study" by Tianlong Chen*, Kaixiong Zhou*, Keyu Duan, Wenqing Zheng, Peihao Wang, Xia Hu, Zhangyang Wang

[NeurIPS 2021] Galerkin Transformer: a linear attention without softmax

Code for IntraQ, PyTorch implementation of our paper under review

The official implementation of the CVPR 2021 paper FAPIS: a Few-shot Anchor-free Part-based Instance Segmenter

Music Classification: Beyond Supervised Learning, Towards Real-world Applications

Human-Pose-and-Motion History

Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

A Python library for working with arbitrary-dimension hypercomplex numbers following the Cayley-Dickson construction of algebras.

A configurable, tunable, and reproducible library for CTR prediction

Demo project for real time anomaly detection using kafka and python

Gym Threat Defense

[Preprint] "Bag of Tricks for Training Deeper Graph Neural Networks A Comprehensive Benchmark Study" by Tianlong Chen, Kaixiong Zhou, Keyu Duan, Wenqing Zheng, Peihao Wang, Xia Hu, Zhangyang Wang