An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Last update: Dec 14, 2022

Related tags

Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
PointConv: Deep Convolutional Networks on 3D Point Clouds

Comments

which version should the `finetuner` module be?
I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

from finetuner.tuner.callback import BestModelCheckpoint from finetuner.tuner.pytorch.losses import TripletLoss from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.
opened by elenacliu 4
fix: fix previous structure

I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

opened by elenacliu 2
Add dataset, preprocess and pointnet2

Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

@numb3r3 @jemmyshin

opened by elenacliu 1
In `farthest_point_sample` function, the first centroid sampled is not random.

I'm reading your codes.

https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

opened by elenacliu 1
Provide diffusion encoder

https://github.com/nmwsharp/diffusion-net

I'm unfamiliar with the jina ecosystem. So this is my first issue.

Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

opened by fire 2

Releases(v0.3.0)

v0.3.0(May 6, 2022)

Source code(tar.gz)
Source code(zip)
v0.2.0(Feb 21, 2022)

Source code(tar.gz)
Source code(zip)

Owner

Jina AI

A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.

GitHub Repository

Glyphtracer is an app for converting images of letters to a font

Glyphtracer takes an image that contains pictures of several letters. It recognizes all them and lets the user tag each letter to a Unicode code point. It then converts the images to vector form and

38 Dec 24, 2022

GPU-accelerated image processing using cupy and CUDA

napari-cupy-image-processing GPU-accelerated image processing using cupy and CUDA This napari plugin was generated with Cookiecutter using with @napar

16 Oct 26, 2022

Simple Python / ImageMagick script to package images into WAD3s for use as GoldSrc textures.

WADs Out For [The] Ladies Simple Python / ImageMagick script to package images into WAD3s for use as GoldSrc textures. Development mostly focused on L

5 Apr 09, 2022

A drop-in replacement for django's ImageField that provides a flexible, intuitive and easily-extensible interface for quickly creating new images from the one assigned to the field.

django-versatileimagefield A drop-in replacement for django's ImageField that provides a flexible, intuitive and easily-extensible interface for creat

490 Dec 13, 2022

Image comparison slider component for Streamlit

Streamlit Image Comparison Component A simple Streamlit Component to compare images with a slider in Streamlit apps using Knightlab's JuxtaposeJS. It

109 Dec 23, 2022

A SIXEL encoder/decoder implementation derived from kmiya's sixel

libsixel What is this? This package provides encoder/decoder implementation for DEC SIXEL graphics, and some converter programs. (https://youtu.be/0Sa

2k Jan 09, 2023

Simple to use image handler for python sqlite3.

SQLite Image Handler Simple to use image handler for python sqlite3. Functions Function Name Parameters Returns init databasePath : str tableName : st

7 Sep 16, 2022

PyLibTiff - a wrapper to the libtiff library to Python using ctypes

PyLibTiff is a package that provides: a wrapper to the libtiff library to Python using ctypes. a pure Python module for reading and writing TIFF and L

105 Dec 21, 2022

The coolest python qrcode maker for small businesses.

QR.ify The coolest python qrcode maker for small businesses. Author Zach Yusuf Project description Python final project. Built to test python skills P

2 Jan 14, 2022

Change the image one color channel at a time.

Building-a-Contact-Sheet This hands-on Project is in Python 3 Programming Specialization offered by University of Michigan via Coursera. change the im

1 Jan 03, 2022

Cat avatars for adult independent users

Cat avatars for adult independent users Samples (Natasha, wake up!) Usage Check values from https://shantichat.github.io/avacats/index.json: { "sizes"

4 Nov 05, 2021

Visage Differentiation is a GUI application for outlining and labeling the visages in an image.

Visage Differentiation Visage Differentiation is a GUI application for outlining and labeling the visages in an image. The main functionality is provi

0 Jan 13, 2022

Generate your own QR Code and scan it to see the results! Never use it for malicious purposes.

QR-Code-Generator-Python Choose the name of your generated QR .png file. If it happens to open the .py file (the application), there are specific comm

1 Dec 23, 2021

CropImage is a simple toolkit for image cropping, detecting and cropping main body from pictures.

CropImage is a simple toolkit for image cropping, detecting and cropping main body from pictures. Support face and saliency detection.

15 Dec 22, 2022

Xmas-Tree-GIF-Tool - Convert any given animated gif file into an animation in GIFT CSV format

This repo is made to participate in Matt Parker's XmasTree 2021 event. Convert a

2 Dec 30, 2021

Computer art based on joining transparent images

Computer Art There is no must in art because art is free. Introduction The following tutorial exaplains how to generate computer art based on a series

12 Jul 30, 2022

Png2Jpg tool will help you convert from png image format to jpg images format.

PNG 2 JPG All codes assume running from root directory. Please update the sys path at the beginning of the codes before running. Over View Png2Jpg too

2 Dec 27, 2021

Find target hash collisions for Apple's NeuralHash perceptual hash function.💣

neural-hash-collider Find target hash collisions for Apple's NeuralHash perceptual hash function. For example, starting from a picture of this cat, we

630 Jan 01, 2023

DP2 graph edit codes.

必要なソフト・パッケージ Python3 Numpy JSON Matplotlib 動作確認環境 MacBook Air M1 Python 3.8.2 (arm64) Numpy 1.22.0 Matplotlib 3.5.1 JSON 2.0.9 使い方 draw_time_histgram(

1 Feb 19, 2022

A tool for making simple-style text posters or wallpapers with high resolution.

PurePoster PurePoster is a fancy tool for making arbitrary-resolution, simple-style posters or wallpapers with text in center. Functionality PurePoste

4 Jul 09, 2022