Differentiable rasterization applied to 3D model simplification tasks

Related tags

Deep Learningnvdiffmodeling
Overview

nvdiffmodeling

Teaser image

Differentiable rasterization applied to 3D model simplification tasks, as described in the paper:

Appearance-Driven Automatic 3D Model Simplification
Jon Hasselgren, Jacob Munkberg, Jaakko Lehtinen, Miika Aittala and Samuli Laine
https://research.nvidia.com/publication/2021-04_Appearance-Driven-Automatic-3D
https://arxiv.org/abs/2104.03989

License

Copyright © 2021, NVIDIA Corporation. All rights reserved.

This work is made available under the Nvidia Source Code License.

For business inquiries, please contact [email protected]

Installation

Requires VS2019+, Cuda 10.2+ and PyTorch 1.6+

Tested in Anaconda3 with Python 3.6 and PyTorch 1.8

One time setup (Windows)

Install the Cuda toolkit (required to build the PyTorch extensions). We support Cuda 10.2 and above. Pick the appropriate version of PyTorch compatible with the installed Cuda toolkit. Below is an example with Cuda 11.1

conda create -n dmodel python=3.6
activate dmodel
conda install pytorch torchvision torchaudio cudatoolkit=11.1 -c pytorch -c conda-forge
conda install imageio
pip install PyOpenGL glfw

Install nvdiffrast: https://github.com/NVlabs/nvdiffrast in the dmodel conda env.

Every new command prompt

activate dmodel

Examples

Sphere to cow example:

python train.py --config configs/spot.json

The results will be stored in the out folder. The Spot model was created and released into the public domain by Keenan Crane.

Additional assets can be downloaded here [205MB]. Unzip and place the subfolders in the project data folder, e.g., data\skull. All assets are copyright of their respective authors, see included license files for further details.

Included examples

  • skull.json - Joint normal map and shape optimization on a skull
  • ewer.json - Ewer model from a reduced mesh as initial guess
  • gardenina.json - Aggregate geometry example
  • hibiscus.json - Aggregate geometry example
  • figure_brushed_gold_64.json - LOD example, trained against a supersampled reference
  • figure_displacement.json - Joint shape, normal map, and displacement map example

The json files that end in _paper.json are configs with the settings used for the results in the paper. They take longer and require a GPU with sufficient memory.

Server usage (through Docker)

  • Build docker image (run the command from the code root folder). docker build -f docker/Dockerfile -t diffmod:v1 . Requires a driver that supports Cuda 10.1 or newer.

  • Start an interactive docker container: docker run --gpus device=0 -it --rm -v /raid:/raid -it diffmod:v1 bash

  • Detached docker: docker run --gpus device=1 -d -v /raid:/raid -w=[path to the code] diffmod:v1 python train.py --config configs/spot.json

Comments
  • Proposition for more comprehensive Readme

    Proposition for more comprehensive Readme

    First of all, thank you for making this freely available for everyone! This is really a great move!

    I wanted to suggest a few things for the readme: 1.) Add "Microsoft Visual C++" to the requirements. It should be clear that you have to tick the C++ development environment while installing VS2019.

    2.) Install Git via "conda install git" (This makes it easier to install nvdiffrast and clone the repo)

    3.) Change "Install NVDiffRast" to something more descriptive: pip install -e git+https://github.com/NVlabs/[email protected]#egg=nvdiffrast

    4.) Add another step to the installation, detailing to move to a desired folder via the Anaconda Prompt and cloning this repo: cd YOUR/PATH/HERE git clone https://github.com/NVlabs/nvdiffmodeling.git and afterwards to move into the folder cd nvdiffmodeling

    5.) Installing CUDA via Anaconda only deploys the bare essential files. The CUDA path is not set when doing it this way. So I downloaded the CUDA toolkit from nvidia and installed it afterwards, to fix that.

    I am sure that all of these things are obvious for professionals, but it took me a while to figure all of this out, so I would like to save other people's time.

    opened by morph3us-net 7
  • About the raymarch fitting case

    About the raymarch fitting case

    Thanks for the marvelous work! I have some questions related to the shadertoy fitting case: It seems figure 22 & figure 23 use some special uv un-wrap methods, can you elaborate the algorithm under the hood? image

    opened by outlawever 4
  • RuntimeError: Error building extension 'renderutils_plugin'

    RuntimeError: Error building extension 'renderutils_plugin'

    I am trying to get this to work to test the workflow but ran into the error I have installed "Cuda" 11.1 and everything else as in the tutorial Graphics accelerator 3080t

    After trying to run a test command, it gives me the following text Do you have any ideas how to fix this? I am more of a technical artist and a cursory search did not return much

    
(dmodel) I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main>python train.py --config configs/spot.json
Using C:\Users\alexp\AppData\Local\torch_extensions\torch_extensions\Cache as PyTorch extensions root...
C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py:305: UserWarning: Error checking compiler version for cl: 'utf-8' codec can't decode byte 0x8e in position 0: invalid start byte
  warnings.warn(f'Error checking compiler version for {compiler}: {error}')
Detected CUDA files, patching ldflags
Emitting ninja build file C:\Users\alexp\AppData\Local\torch_extensions\torch_extensions\Cache\renderutils_plugin\build.ninja...
Building extension module renderutils_plugin...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
[1/7] cl /showIncludes -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\common.cpp /Focommon.o
FAILED: common.o
cl /showIncludes -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\common.cpp /Focommon.o
Оптимизирующий компилятор Microsoft (R) C/C++ версии 19.25.28611 для x64
(C) Корпорация Майкрософт (Microsoft Corporation).  Все права защищены.

Примечание: включение файла:  C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\cuda_runtime.h
Примечание: включение файла:   C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_config.h
Примечание: включение файла:   C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\builtin_types.h
Примечание: включение файла:    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\device_types.h
Примечание: включение файла:     C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_defines.h
Примечание: включение файла:    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\driver_types.h
Примечание: включение файла:     C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_defines.h
Примечание: включение файла:     C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\vector_types.h
Примечание: включение файла:      C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\crt/host_defines.h
Примечание: включение файла:     C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\limits.h
Примечание: включение файла:      C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\vcruntime.h
Примечание: включение файла:       C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\sal.h
Примечание: включение файла:        C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\concurrencysal.h
Примечание: включение файла:       C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\vadefs.h
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include\driver_types.h(78): fatal error C1083: Не удается открыть файл включение: stddef.h: No such file or directory,
[2/7] cl /showIncludes -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\torch_bindings.cpp /Fotorch_bindings.o
FAILED: torch_bindings.o
cl /showIncludes -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 /MD /wd4819 /wd4251 /wd4244 /wd4267 /wd4275 /wd4018 /wd4190 /EHsc -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\torch_bindings.cpp /Fotorch_bindings.o
Оптимизирующий компилятор Microsoft (R) C/C++ версии 19.25.28611 для x64
(C) Корпорация Майкрософт (Microsoft Corporation).  Все права защищены.

Примечание: включение файла:  C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch/extension.h
Примечание: включение файла:   C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include\torch/all.h
Примечание: включение файла:    C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include\torch/cuda.h
Примечание: включение файла:     C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch/csrc/WindowsTorchApiMacro.h
Примечание: включение файла:      C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\c10/macros/Export.h
Примечание: включение файла:       C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\c10/macros/cmake_macros.h
Примечание: включение файла:     C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\cstdint
Примечание: включение файла:      C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\yvals.h
Примечание: включение файла:       C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\yvals_core.h
Примечание: включение файла:        C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\vcruntime.h
Примечание: включение файла:         C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\sal.h
Примечание: включение файла:          C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\concurrencysal.h
Примечание: включение файла:         C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\vadefs.h
Примечание: включение файла:        C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\xkeycheck.h
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\yvals.h(12): fatal error C1083: Не удается открыть файл включение: crtdbg.h: No such file or directory,
[3/7] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output mesh.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\mesh.cu -o mesh.cuda.o
FAILED: mesh.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output mesh.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\mesh.cu -o mesh.cuda.o
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\crtdefs.h(10): fatal error C1083: ═х єфрхЄё  юЄъЁ√Є№ Їрщы тъы■ўхэшх: corecrt.h: No such file or directory,
mesh.cu
[4/7] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output loss.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\loss.cu -o loss.cuda.o
FAILED: loss.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output loss.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\loss.cu -o loss.cuda.o
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\crtdefs.h(10): fatal error C1083: ═х єфрхЄё  юЄъЁ√Є№ Їрщы тъы■ўхэшх: corecrt.h: No such file or directory,
loss.cu
[5/7] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output bsdf.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\bsdf.cu -o bsdf.cuda.o
FAILED: bsdf.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output bsdf.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\bsdf.cu -o bsdf.cuda.o
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\crtdefs.h(10): fatal error C1083: ═х єфрхЄё  юЄъЁ√Є№ Їрщы тъы■ўхэшх: corecrt.h: No such file or directory,
bsdf.cu
[6/7] C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output normal.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\normal.cu -o normal.cuda.o
FAILED: normal.cuda.o
C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\bin\nvcc --generate-dependencies-with-compile --dependency-output normal.cuda.o.d -Xcudafe --diag_suppress=dll_interface_conflict_dllexport_assumed -Xcudafe --diag_suppress=dll_interface_conflict_none_assumed -Xcudafe --diag_suppress=field_without_dll_interface -Xcudafe --diag_suppress=base_class_has_different_dll_interface -Xcompiler /EHsc -Xcompiler /wd4190 -Xcompiler /wd4018 -Xcompiler /wd4275 -Xcompiler /wd4267 -Xcompiler /wd4244 -Xcompiler /wd4251 -Xcompiler /wd4819 -Xcompiler /MD -DTORCH_EXTENSION_NAME=renderutils_plugin -DTORCH_API_INCLUDE_EXTENSION_H -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\torch\csrc\api\include -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\TH -IC:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\include\THC "-IC:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.1\include" -IC:\ProgramData\Anaconda3\envs\dmodel\Include -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_BFLOAT16_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_86,code=compute_86 -gencode=arch=compute_86,code=sm_86 -c I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\c_src\normal.cu -o normal.cuda.o
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.25.28610\include\crtdefs.h(10): fatal error C1083: ═х єфрхЄё  юЄъЁ√Є№ Їрщы тъы■ўхэшх: corecrt.h: No such file or directory,
normal.cu
ninja: build stopped: subcommand failed.
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1672, in _run_ninja_build
    env=env)
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "train.py", line 20, in <module>
    import src.renderutils as ru
  File "I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\__init__.py", line 9, in <module>
    from .ops import xfm_points, xfm_vectors, image_loss, prepare_shading_normal, lambert, pbr_specular, pbr_bsdf, _fresnel_shlick, _ndf_ggx, _lambda_ggx, _masking_smith
  File "I:\Python\Nvidia_Lod\nvdiffmodeling-main\nvdiffmodeling-main\src\renderutils\ops.py", line 61, in <module>
    torch.utils.cpp_extension.load(name='renderutils_plugin', sources=source_paths, extra_ldflags=ldflags, with_cuda=True, verbose=True)
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1092, in load
    keep_intermediates=keep_intermediates)
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1303, in _jit_compile
    is_standalone=is_standalone)
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1408, in _write_ninja_file_and_build_library
    error_prefix=f"Error building extension '{name}'")
  File "C:\ProgramData\Anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1682, in _run_ninja_build
    raise RuntimeError(message) from e
RuntimeError: Error building extension 'renderutils_plugin'
    opened by TheAlexPG 4
  • about effect Fig.22 and Fig 23

    about effect Fig.22 and Fig 23

    I read your paper and video in detail，about LEARNING MESH AND MATERIALS FROM IMPLICIT SURFACES,How to improve the effect？ Can you give me some suggestions on setting super parameters? I used my own model as the target, started from the sphere you provided, and carried out experiments. I found that the feet and face were always unsatisfactory? There is a big gap with the rendering of snails and elephants mentioned in your paper. can you give me some adivce. image

    opened by Lucklycat 2
  • Texture maps have a lot of noise

    Texture maps have a lot of noise

    Thank you very much for your work！ When I tried the ewer example, I found the following textures： image When you zoom in, you will find many red, yellow and green noises image This is still the case when I import the output obj into blender, but the contrast image rendered from nvdiffast does not show this situation. What is the reason? image

    opened by Lucklycat 2
  • How to improve the optimization effect

    How to improve the optimization effect

    Your work is great. Thank you very much for your openness. I tried to use rendering on my own model, but I encountered the following problems and would like to consult: 1.In the process of rendering, I found that the face of the character, especially the eyes, was not poorly optimized, and the color could not be reflected. Are there any parameters that need to be adjusted? image 2.During rendering, I encountered that. mtl contains a map_ The texture of d cannot be represented in the renderer. How can I solve this problem? image

    Thank you for your reply

    opened by Lucklycat 2
  • Lower training resolution => heavier texture pixelation. But why?

    Lower training resolution => heavier texture pixelation. But why?

    Hi, and thanks for great paper and comprehensive code! Could you please help me understand one thing:

    I'm running configs/dancer_displacement.json. Despite "texture_res" being 2048, these 2048 x 2048 textures "learn" huge pixels at lower "train_res":

    image

    And this isn't just poor interpolation in Blender, but these are actual large solid blocks (of different scale!) in normal_map_opt and kd_map_opt! Here is a part of texture_n.png from the top left experiment, 1:1 scale:

    image

    Why do these maps get more pixelated instead of getting more blurry? I've spent couple days searching the code for the source of this behaviour, and I believe there are only three relevant lines (for now let's consider texture only):

    Interpolation doesn't use 'nearest' method so it shouldn't be the source of big pixels. And because views are always different, rasterization should be a problem too. But probably I'm just missing some rendering subtleties?

    Thanks again!

    opened by shrubb 2
  • msvcrt.lib(loadcfg.obj) : error LNK2001: unresolved external symbol __enclave_config

    msvcrt.lib(loadcfg.obj) : error LNK2001: unresolved external symbol __enclave_config

    hi, Thanks for your project. When I try to run sphere to cow demo, I will report this error: msvcrt.lib(loadcfg.obj) : error LNK2001: unresolved external symbol __enclave_config renderutils_plugin.pyd : fatal error LNK1120: Unresolved external command ninja: build stopped: subcommand failed.

    Is it because of my Nanjia version? my current version is: nanjia:1.10.2 pytorch:1.8.1 torchvision:0.9.1 cudnn:7.6.5 cuda:10.2

    opened by myshiop 2
  • matrix gradients does not flow back when using renderutils xfm_points

    matrix gradients does not flow back when using renderutils xfm_points

    hi, when using renderutils.xfm_points (e.g. in render.py v_pos_clip = ru.xfm_points(mesh.v_pos[None, ...], mtx_in) - and trying to optimize camera pose in mtx_in - gradients does not flow back. Only when using use_python=True it worked.

    opened by maorp 1
  • using camera calibration matrices

    using camera calibration matrices

    Hi, first thanks for sharing this work! I'm trying to solve inverse rendering problem using a photogrammetry dataset which comes with accurate camera calibration mats. i have started from the spot example and i need to convert my intrinsic\extrinsic params to be used in the rastering and cant find an example or a guide to do it. the only hint is that I know it should be converted to openGL and NDC according to nvdiffrast, ill be happy to get an advice, thanks.

    opened by maorp 1
  • some question about the complex material, such as that ray marching snail

    some question about the complex material, such as that ray marching snail

    Hi, Thank you very much for your research. I'm trying to use your framework to approximate complex materials, but the blender complex material dataset I made with my own is not very good, because I encountered many problems with camera and light settings in my own rendering, Is the ray marching snail used in this paper directly rendered with shadertoy, or is the ray marching process written in Python?

    opened by myshiop 1
  • where can i find the Animation and Skinning demo

    where can i find the Animation and Skinning demo

    hi, i am read the papaer--Appearance-Driven Automatic 3D Model Simplification, think it's great. But in the demo,i can not find the Animation and Skinning demo, where can i find it ? I want to study how to do it. please help ! thanks

    opened by Lucklycat 2
  • CUDA out of memory.

    CUDA out of memory.

    RuntimeError: CUDA out of memory. Tried to allocate 130.00 MiB (GPU 0; 8.00 GiB total capacity; 7.11 GiB already allocated; 0 bytes free; 7.30 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

    opened by d-sharafeldeen 2
  • How to test on custom .obj file?

    How to test on custom .obj file?

    I am trying to test model simplification on my own .obj. I have created a .mtl file for the .obj but I got an error. How to export .obj file to train the model?

    opened by sanbuddhacharyas 0
  • Unable to execute the program, error during cpp_extension.py running at start

    Unable to execute the program, error during cpp_extension.py running at start 

    (dmodel) D:\NVIDIATools\nvdiffmodeling-main>python train.py --config configs/spot.json
No CUDA runtime is found, using CUDA_HOME='C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.5'
Using C:\Users\redacted\AppData\Local\torch_extensions\torch_extensions\Cache\py36_cpu as PyTorch extensions root...
Detected CUDA files, patching ldflags
Emitting ninja build file C:\Users\redacted\AppData\Local\torch_extensions\torch_extensions\Cache\py36_cpu\renderutils_plugin\build.ninja...
Traceback (most recent call last):
  File "train.py", line 20, in <module>
    import src.renderutils as ru
  File "D:\NVIDIATools\nvdiffmodeling-main\src\renderutils\__init__.py", line 9, in <module>
    from .ops import xfm_points, xfm_vectors, image_loss, prepare_shading_normal, lambert, pbr_specular, pbr_bsdf, _fresnel_shlick, _ndf_ggx, _lambda_ggx, _masking_smith
  File "D:\NVIDIATools\nvdiffmodeling-main\src\renderutils\ops.py", line 61, in <module>
    torch.utils.cpp_extension.load(name='renderutils_plugin', sources=source_paths, extra_ldflags=ldflags, with_cuda=True, verbose=True)
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1136, in load
    keep_intermediates=keep_intermediates)
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1347, in _jit_compile
    is_standalone=is_standalone)
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1445, in _write_ninja_file_and_build_library
    is_standalone=is_standalone)
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1834, in _write_ninja_file_to_build_library
    cuda_flags = common_cflags + COMMON_NVCC_FLAGS + _get_cuda_arch_flags()
  File "C:\Users\redacted\anaconda3\envs\dmodel\lib\site-packages\torch\utils\cpp_extension.py", line 1606, in _get_cuda_arch_flags
    arch_list[-1] += '+PTX'
IndexError: list index out of range

(dmodel) D:\NVIDIATools\nvdiffmodeling-main>

    After installing the prerequisites, only difference to the instruction is CUDA 11.5 (and i changed the version to the pip install line to that same version) Getting the above when attempting to run the example spot cow training.

    Something i'm doing wrong, incompatibility with versions or something else entirely?

    opened by RiversJohn 3
Releases(v1.0)
Owner
NVIDIA Research Projects
NVIDIA Research Projects
GitHub Repository
Python project to take sound as input and output as RGB + Brightness values suitable for DMX

sound-to-light Python project to take sound as input and output as RGB + Brightness values suitable for DMX Current goals: Get one pixel working: Vary

Bobby Cox 1 Nov 17, 2021
Code release for Local Light Field Fusion at SIGGRAPH 2019

Local Light Field Fusion Project | Video | Paper Tensorflow implementation for novel view synthesis from sparse input images. Local Light Field Fusion

1.1k Dec 27, 2022
DenseNet Implementation in Keras with ImageNet Pretrained Models

DenseNet-Keras with ImageNet Pretrained Models This is an Keras implementation of DenseNet with ImageNet pretrained weights. The weights are converted

Felix Yu 568 Oct 31, 2022
iNAS: Integral NAS for Device-Aware Salient Object Detection

iNAS: Integral NAS for Device-Aware Salient Object Detection Introduction Integral search design (jointly consider backbone/head structures, design/de

顾宇超 77 Dec 02, 2022
Animal Sound Classification (Cats Vrs Dogs Audio Sentiment Classification)

this is a simple artificial neural network model using deep learning and torch-audio to classify cats and dog sounds.

crispengari 3 Dec 05, 2022
Pytorch code for semantic segmentation using ERFNet

ERFNet (PyTorch version) This code is a toolbox that uses PyTorch for training and evaluating the ERFNet architecture for semantic segmentation. For t

Edu 394 Jan 01, 2023
Fashion Entity Classification

Fashion-Entity-Classification - Fashion-MNIST is a dataset of Zalando's article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grays

ADITYA SHAH 1 Jan 04, 2022
Syntax-Aware Action Targeting for Video Captioning

Syntax-Aware Action Targeting for Video Captioning Code for SAAT from "Syntax-Aware Action Targeting for Video Captioning" (Accepted to CVPR 2020). Th

59 Oct 13, 2022
A Bayesian cognition approach for belief updating of correlation judgement through uncertainty visualizations

Overview Code and supplemental materials for Karduni et al., 2020 IEEE Vis. "A Bayesian cognition approach for belief updating of correlation judgemen

Ryan Wesslen 1 Feb 08, 2022
Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks

Stochastic Downsampling for Cost-Adjustable Inference and Improved Regularization in Convolutional Networks (SDPoint) This repository contains the cod

Jason Kuen 17 Jul 04, 2022
Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network"

M3D-VTON: A Monocular-to-3D Virtual Try-On Network Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network" Paper | Suppl

109 Dec 29, 2022
Autoencoder - Reducing the Dimensionality of Data with Neural Network

autoencoder Implementation of the Reducing the Dimensionality of Data with Neural Network – G. E. Hinton and R. R. Salakhutdinov paper. Notes Aim to m

Jordan Burgess 13 Nov 17, 2022
An educational AI robot based on NVIDIA Jetson Nano.

JetBot Looking for a quick way to get started with JetBot? Many third party kits are now available! JetBot is an open-source robot based on NVIDIA Jet

NVIDIA AI IOT 2.6k Dec 29, 2022
Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020)

Causality In Traffic Accident (Under Construction) Repository for Traffic Accident Benchmark for Causality Recognition (ECCV 2020) Overview Data Prepa

Tackgeun 21 Nov 20, 2022
Instance Segmentation by Jointly Optimizing Spatial Embeddings and Clustering Bandwidth

Instance segmentation by jointly optimizing spatial embeddings and clustering bandwidth This codebase implements the loss function described in: Insta

209 Dec 07, 2022
Air Quality Prediction Using LSTM

AirQualityPredictionUsingLSTM In this Repo, i present to you the winning solution of smart gujarat hackathon 2019 where the task was to predict the qu

Deepak Nandwani 2 Dec 13, 2022
Efficient face emotion recognition in photos and videos

This repository contains code of face emotion recognition that was developed in the RSF (Russian Science Foundation) project no. 20-71-10010 (Efficien

Andrey Savchenko 239 Jan 04, 2023
Sentiment analysis translations of the Bhagavad Gita

Sentiment and Semantic Analysis of Bhagavad Gita Translations It is well known that translations of songs and poems not only breaks rhythm and rhyming

Machine learning and Bayesian inference @ UNSW Sydney 3 Aug 01, 2022
A project to make Amazon Echo respond to sign language using your webcam

Making Alexa respond to Sign Language using Tensorflow.js Try the live demo Read the Blog Post on Tensorflow's Blog Coming Soon Watch the video This p

Abhishek Singh 444 Jan 03, 2023
Election Exit Poll Prediction and U.S.A Presidential Speech Analysis using Machine Learning

Machine_Learning Election Exit Poll Prediction and U.S.A Presidential Speech Analysis using Machine Learning This project is based on 2 case-studies:

Avnika Mehta 1 Jan 27, 2022