OpenCV-Erlang/Elixir bindings

Related tags

Computer Visionopencvelixirelixir-libraryerlang-library
Overview

evision [WIP]

: OS : arch Build Status
Ubuntu 20.04 arm64 CI
Ubuntu 20.04 armv7 CI
Ubuntu 20.04 s390x CI
Ubuntu 20.04 ppc64le CI
Ubuntu 20.04 x86_64 CI
macOS 11 Big Sur x86_64 CI

Apple Silicon M1-series are supported, actually I coded and tested this project on my M1 Max MacBook Pro, but M1 GitHub Action runners are not yet available.

Nerves Support

Nerves

Prebuilt firmwares are available here. Select the most recent run and scroll down to the Artifacts section, download the firmware file for your board and run

fwup /path/to/the/downloaded/firmware.fw

SSH keys can be found in nerves/id_rsa[.pub]. For obvious security reason, please use these prebuilt firmwares for evaluation only.

Description

evision will pull OpenCV source code from GitHub, then parse and automatically generate corresponding OpenCV-Elixir bindings.

This project uses and modifies gen2.py and hdr_parser.py from the python module in the OpenCV repo so that they output header files that can be used in Elixir bindings.

We hope this project can largely reduce the work of manually porting OpenCV functions/modules to Elixir.

Current available modules:

  • calib3d
  • core
  • features2d
  • flann
  • highgui
  • imgcodecs
  • imgproc
  • ml
  • photo
  • stitching
  • ts
  • video
  • videoio

Note, edit config/config.exs to enable/disable OpenCV modules and image coders.

Dependencies

Required

  • Python3 (Only during the compilation, to generate binding files)
  • CMake
  • Erlang development library/headers.

Optional

  • curl/wget. To download OpenCV source zip file.

    Optional if you put the source zip file to 3rd_party/cache/opencv-${OPENCV_VER}.zip.

  • unzip. To unzip the OpenCV source zip file.

    Optional if you supply OpenCV source code at 3rd_party/opencv/opencv-${OPENCV_VER}.

Installation

In order to use evision, you will need Elixir installed. Then create an Elixir project via the mix build tool:

$ mix new my_app

Then you can add evision as dependency in your mix.exs. At the moment you will have to use a Git dependency while we work on our first release:

def deps do
  [
    {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
  ]
end

Note

  • Use MAKE_BUILD_FLAGS="-j$(nproc)" environment variable to set number of jobs for compiling.

    Default value: "-j#{System.schedulers_online()}". In mix.exs.

  • Use TOOLCHAIN_FILE="/path/to/toolchain.cmake" to set your own toolchain.

    Default value: "nerves/toolchain.cmake". In Makefile.

  • Edit config/config.exs to enable/disable OpenCV modules and image coders.

  • Some useful commands

    MIX_ENV=dev
OPENCV_VER=4.5.4
MIX_TARGET=rpi4

# delete OpenCV related CMake build caches.
rm -rf "_build/${MIX_ENV}/lib/evision/cmake_opencv_${OPENCV_VER}"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/cmake_opencv_${OPENCV_VER}"

# remove downloaded OpenCV source zip file.
rm -f "3rd_party/cache/opencv-${OPENCV_VER}"

# delete evision.so (so that `make` can rebuild it, useful when you manually modified C/C++ source code)
rm -f "_build/${MIX_ENV}/lib/evision/priv/evision.so"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/priv/evision.so"

# delete evision related CMake build caches.
rm -rf "_build/${MIX_ENV}/lib/evision/cmake_evision"
## for nerves
rm -rf "_build/${MIX_TARGET}_${MIX_ENV}/lib/evision/cmake_evision"

Current Status

{:ok, gray_mat} = OpenCV.imread("/path/to/img.png", flags: OpenCV.cv_imread_grayscale)
{:ok, gray_blur_mat} = OpenCV.blur(gray_mat, [10,10], anchor: [1,1])
{:ok, colour_mat} = OpenCV.imread("/path/to/img.png")
{:ok, colour_blur_mat} = OpenCV.blur(colour_mat, [10,10], anchor: [1,1])
:ok = OpenCV.imwrite("/path/to/img-gray-and-blur.png", gray_blur_mat)
:ok = OpenCV.imwrite("/path/to/img-colour-and-blur.png", colour_blur_mat)

{:ok, cap} = OpenCV.VideoCapture.videocapture(0)
{:ok, cap_mat} = OpenCV.VideoCapture.read(cap)
:ok = OpenCV.imwrite("/path/exists/capture-mat.png", cap_mat)
:error = OpenCV.imwrite("/path/not/exists/capture-mat.png", cap_mat)

Todo

  • Update .py files in py_src so that they output header files for Erlang bindings.

  • Automatically generate erl_cv_nif.ex.

  • Automatically generate opencv_*.ex files using Python.

  • Automatically convert enum constants in C++ to "constants" in Elixir

  • When a C++ function's return value's type is bool, map true to :ok and false to :error.

    # not this
{:ok, true} = OpenCV.imwrite("/path/to/save.png", mat, [])
# but this
:ok = OpenCV.imwrite("/path/to/save.png", mat, [])

  • Make optional parameters truly optional.

    # not this
{:ok, cap} = OpenCV.VideoCapture.videocapture(0, [])
# but this
{:ok, cap} = OpenCV.VideoCapture.videocapture(0)
# this also changes the above example to
:ok = OpenCV.imwrite("/path/to/save.png", mat)

  • Nerves support (rpi4 for now).

  • Add OpenCV.Mat module.

    {:ok, type} = OpenCV.Mat.type(mat)
{:ok, {:u, 8}}

{:ok, {height, weight, channel}} = OpenCV.Mat.shape(mat)
{:ok, {1080, 1920, 3}}
{:ok, {height, weight}} = OpenCV.Mat.shape(gray_mat)
{:ok, {1080, 1920}}

{:ok, bin_data} = OpenCV.Mat.to_binary(mat)
{:ok, << ... binary data ... >>}

  • Add support for Nx.

    nx_tensor = OpenCV.Mat.to_nx(mat)
#Nx.Tensor<
   u8[1080][1920][3]
   [[ ... pixel data ... ]]
>
{:error, reason} = OpenCV.Mat.to_nx(invalid_mat)

  • Edit config/config.exs to enable/disable OpenCV modules and image coders.

  • Add tests.

How does this work?

  1. This project will first pull OpenCV source code from git (as git submodules).

  2. Inside the OpenCV project, there is an opencv-python module, 3rd_party/opencv/modules/python. If the opencv-python module is enabled,

    cmake ...
    -D PYTHON3_EXECUTABLE=$(PYTHON3_EXECUTABLE) \
    ...

    It will generate files for opencv-python bindings in cmake build dir, cmake_build_dir/modules/python_bindings_generator.

    We are interested in the headers.txt file as it tells us which headers should we parse (this header list changes based on the enabled modules).

    We also need to check another file, pyopencv_custom_headers.h. This file includes pyopencv compatible headers from modules that need special handlings to enable interactions with Python. We will talk about this later.

  3. Originally, the headers.txt file will be passed to 3rd_party/opencv/modules/python/src2/gen2.py and that Python script will then generate pyopencv_*.h in cmake_build_dir/modules/python_bindings_generator. Here we copy that Python script and modify it so that it outputs c_src/evision_*.h files which use c_src/erlcompat.hpp and c_src/nif_utils.hpp to make everything compatible with Erlang.

  4. c_src/opencv.cpp includes almost all specialised and generic evision_to and evision_from functions. They are used for making conversions between Erlang and C++. Some conversion functions are defined in module custom headers.

  5. This is where we need to make some changes to pyopencv_custom_headers.h. We first copy it to c_src/evision_custom_headers.h and copy every file it includes to c_src/evision_custom_headers/. Then we make corresponding changes to c_src/evision_custom_headers/*.hpp files so that these types can be converted from and to Erlang terms. The header include path in c_src/evision_custom_headers.h should be changed correspondingly.

  6. However, it is hard to do step 5 automatically. We can try to create a PR which puts these changed files to the original OpenCV repo's {module_name}/mics/erlang/ directory. Now we just manually save them in c_src/evision_custom_headers. Note that step 5 and 6 are done manually, calling py_src/gen2.py will not have effect on c_src/evision_custom_headers.h and *.hpp files in c_src/evision_custom_headers.

  7. Another catch is that, while function overloading is easy in C++ and optional arguments is simple in Python, they are not quite friendly to Erlang/Elixir. There is basically no function overloading in Erlang/Elixir. Although Erlang/Elixir support optional argument (default argument), it also affects the function's arity and that can be very tricky to deal with. For example,

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts \\ []), do: :nil
  def open(self, filename), do: :nil
  # ... other functions ...
end

    In this case, def open(self, camera_index, opts \\ []), do: :nil will define open/3 and open/2 at the same time. This will cause conflicts with def open(self, filename), do: :nil which defines open/2.

    So we cannot use default arguments. Now, say we have

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts), do: :nil
  def open(self, filename, opt), do: :nil
end

    Function overloading in C++ is relatively simple as compiler does that for us. In this project, we have to do this ourselves in the Erlang/Elixir way. For the example above, we can use guards.

    defmodule OpenCV.VideoCapture do
  def open(self, camera_index, opts) when is_integer(camera_index) do
    # do something
  end
  def open(self, filename, opt) when is_binary(filename) do
    # do something
  end
end

    But there are some cases we cannot distinguish the argument type in Erlang/Elixir becauase they are resources (instance of a certain C++ class).

    defmodule OpenCV.SomeModule do
  # @param mat: Mat
  def func_name(mat) do
    # do something
  end

  # @param mat: UMat
  def func_name(mat) do
    # do something
  end
end

    In such cases, we only keep one definition. The overloading will be done in c_src/opencv.cpp (by evision_to).

  8. Enum handling. Originally, PythonWrapperGenerator.add_const in py_src/gen2.py will be used to handle those enum constants. They will be saved to a map with the enum's string representation as the key, and, of course, enum's value as the value. In Python, when a user uses the enum, say cv2.COLOR_RGB2BGR, it will perform a dynamic lookup which ends up calling corresponding evision_[to|from]. evision_[to|from] will take the responsibility to convert between the enum's string representation and its value. Although in Erlang/Elixir we do have the ability to both create atoms and do the similar lookups dynamically, the problem is that, if an enum is used as one of the arguments in a C++ function, it may be written as void func(int enum) instead of void func(ENUM_TYPE_NAME enum). However, to distinguish between overloaded functions, some types (int, bool, string, char, vector) will be used for guards. For example, void func(int enum) will be translated to def func(enum) when is_integer(enum), do: :nil. Adding these guardians help us to make some differences amongst overloaded functions in step 7. However, that prevents us froming passing an atom to def func(enum) when is_integer(enum), do: :nil. Technically, we can add one more variant def func(enum) when is_atom(enum), do: :nil for this specific example, but there are tons of functions has one or more ints as their input arguments, which means the number of variants in Erlang will increase expoentially (for each int in a C++ function, it can be either a real int or an enum). Another way is just allow it to be either an integer or an atom:

    def func(enum) when is_integer(enum) or is_atom(enum) do
  :nil
end

    But in this way, atoms are created on-the-fly, users cannot get code completion feature for enums from their IDE. But, finally, we have a good news that, in Erlang/Elixir, when a function has zero arguments, you can write its name without explictly calling it, i.e.,

    defmodule M do
  def enum_name(), do: 1
end

1 = M.enum_name
1 = M.enum_name()

    So, in this project, every enum is actually transformed to a function that has zero input arguments.

How do I make it compatible with more OpenCV modules?

Because of the reason in step 6, when you enable more modules, if that module has specialised custom header for python bindings, the custom headers will be added to cmake_build_dir/modules/python_bindings_generator/pyopencv_custom_headers.h. Then you can manually copy corresponding specialised custom headers to c_src/evision_custom_headers and modify these conversion functions in them.

Acknowledgements

  • gen2.py, hdr_parser.py and c_src/erlcompat.hpp were directly copied from the python module in the OpenCV repo. Changes applied.
  • Makefile, CMakeLists.txt and c_src/nif_utils.hpp were also copied from the torchx module in the elixir-nx repo. Minor changes applied.
Comments
  • Proposal: bridging with Nx

    Proposal: bridging with Nx

    I'm developing Excv, a bridge between Nx and OpenCV: https://github.com/zeam-vm/excv

    I hope that evision will be integrated with such bridging. I'd like to contribute to evision if you wish so. Would you please inform me where the #Reference structure includingMat?

    opened by zacky1972 14
  • Share image data between Vix and eVision (and Nx)

    Share image data between Vix and eVision (and Nx)

    I am the author of the Image library that is only possible because of @akash-akya's Vix which is a binding to libvips. Given Vix and eVision have similar problem domains, and very similar matrix implementations, I would like to be able to share image data between the two libraries. This would allow Image (and of course any other library) to leverage the capabilities of both Vix and eVision (and very desirably Nx) in a very memory efficient and performant manner.

    Objectives

    1. Share image data without copying the matrix (I think a shared interpretation is possible) - but need to make sure we know who has memory ownership and who is managing the memory (and I'm not good at this part).
    2. Having a way to agree and share data formats (ie a common lexicon to refer to image data type and matrix shape - perhaps using the Nx terminology?)
    3. Act as a model of how to do the same integration with Nx. I know eVision has Nx integration but it appears to be based upon converting to and from Elixir binaries - which I'd prefer not to do since lots of NIF copying would contradict the memory efficiency and performance objectives.

    I'm looking for guidance, suggestions and ideas - and hopefully someone motivated to tackle this because its a bit beyond my competence. :-)

    Given the intent to also provide integration with Nx, any comments from @josevalim would also be very welcome.

    enhancement 
    opened by kipcole9 13
  • Import Nx tensor that is in {width, height, channels} shape with RGB data format

    Import Nx tensor that is in {width, height, channels} shape with RGB data format

    Cocoa, I have tried everything I can think of to take an Nx tensor that is of the shape {width, height, channels} that contains RGB format data, and convert it in eVision to {height, width, channels} in BGR format.

    Any chance I might ask for your advice and recommendations?

    What I've tried

    Here is what I have tried which is, I think, the required process but the saved image is definitely not what is expected!

    tensor = File.read!("path_to/color_checker.etf") |> :erlang.binary_to_term()
{:ok, mat} = Evision.Nx.to_mat(tensor)
{:ok, transposed} = Evision.Mat.transpose(mat, [1, 0, 2])
{:ok, bgr} = Evision.cvtColor(transposed, Evision.cv_COLOR_RGB2BGR())
Evision.imwrite "some_path/color_checker.jpg", bgr

    I have followed the converse process in Image and the results line up with expectations. Which is not too surprising since Image expects data in {width, height, channels} and RGB format. I mention this just to note that I have verified that the tensor does represent the underlying example image.

    Artifacts

    The image converted to a tensor and stored with :erlang.term_to_binary/1 and zipped: color_checker.etf.zip

    The original image: color_checker

    opened by kipcole9 10
  • cropping of image using ROI not working

    cropping of image using ROI not working

    Please refer to the below images to understand about the issue. This is a function used to get image: Screenshot from 2022-10-10 10-50-12 This is the error window: Screenshot from 2022-10-10 10-52-21 Its working in python for reference: Screenshot from 2022-10-10 10-52-55

    opened by navrkr 9
  • Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

    Building OpenCV is failed when describing evision in deps of mix.exs of a new mix project

    Hi,

    I did the following steps:

    mix new evision_test
cd evision_test

    Then, I added evision to mix.exs as follows:

      defp deps do
    [
      {:evision, "~> 0.1.0-dev", github: "cocoa-xu/evision", branch: "main"}
    ]
  end

    Then, I did the following steps:

    mix deps.get
mix compile

    Then, I got the following error:

    ==> evision
[/Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip]
  End-of-central-directory signature not found.  Either this file is not
  a zipfile, or it constitutes one disk of a multi-part archive.  In the
  latter case the central directory and zipfile comment will be found on
  the last disk(s) of this archive.
unzip:  cannot find zipfile directory in one of /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip or
        /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.zip, and cannot find /Users/zacky/evision_test/deps/evision/3rd_party/cache/opencv-4.5.5.zip.ZIP, period.
make: *** [/Users/zacky/evision_test/deps/evision/3rd_party/opencv/opencv-4.5.5/modules/core/include/opencv2/core/utils/configuration.private.hpp] Error 9
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    Then, I did the following steps:

    mix deps.compile evision

    Then, I got the following fatal error:

    ==> evision
-- The CXX compiler identification is AppleClang 13.0.0.13000029
-- The C compiler identification is AppleClang 13.0.0.13000029
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
(snip)
-- General configuration for OpenCV 4.5.5 =====================================
--   Version control:               ca737c9
-- 
--   Platform:
--     Timestamp:                   2022-01-26T09:38:53Z
--     Host:                        Darwin 21.2.0 arm64
--     CMake:                       3.22.1
--     CMake generator:             Unix Makefiles
--     CMake build tool:            /usr/bin/make
--     Configuration:               RELEASE
-- 
--   CPU/HW features:
--     Baseline:                    NEON FP16
-- 
--   C/C++:
--     Built as dynamic libs?:      YES
--     C++ standard:                11
--     C++ Compiler:                /Library/Developer/CommandLineTools/usr/bin/c++  (ver 13.0.0.13000029)
--     C++ flags (Release):         -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C++ flags (Debug):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     C Compiler:                  /Library/Developer/CommandLineTools/usr/bin/cc
--     C flags (Release):           -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
--     C flags (Debug):             -DPNG_ARM_NEON_OPT=0   -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wno-delete-non-virtual-dtor -Wno-unnamed-type-template-args -Wno-comment -fdiagnostics-show-option -Qunused-arguments -Wno-semicolon-before-method-body -ffunction-sections -fdata-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
--     Linker flags (Release):      -Wl,-dead_strip  
--     Linker flags (Debug):        -Wl,-dead_strip  
--     ccache:                      NO
--     Precompiled headers:         NO
--     Extra dependencies:
--     3rdparty dependencies:
-- 
--   OpenCV modules:
--     To be built:                 calib3d core dnn features2d flann highgui imgcodecs imgproc ml objdetect photo python2 stitching ts video videoio
--     Disabled:                    gapi world
--     Disabled by dependency:      -
--     Unavailable:                 java python3
--     Applications:                perf_tests apps
--     Documentation:               NO
--     Non-free algorithms:         NO
-- 
--   GUI:                           COCOA
--     Cocoa:                       YES
--     VTK support:                 NO
-- 
--   Media I/O: 
--     ZLib:                        zlib (ver 1.2.11)
--     JPEG:                        build-libjpeg-turbo (ver 2.1.2-62)
--     WEBP:                        build (ver encoder: 0x020f)
--     PNG:                         libpng (ver 1.6.37)
--     TIFF:                        libtiff (ver 42 / 4.2.0)
--     JPEG 2000:                   build (ver 2.4.0)
--     OpenEXR:                     OpenEXR::OpenEXR (ver 3.1.3)
--     HDR:                         YES
--     SUNRASTER:                   YES
--     PXM:                         YES
--     PFM:                         YES
-- 
--   Video I/O:
--     DC1394:                      NO
--     FFMPEG:                      YES
--       avcodec:                   YES (58.134.100)
--       avformat:                  YES (58.76.100)
--       avutil:                    YES (56.70.100)
--       swscale:                   YES (5.9.100)
--       avresample:                YES (4.0.0)
--     GStreamer:                   NO
--     AVFoundation:                YES
-- 
--   Parallel framework:            GCD
-- 
--   Trace:                         YES (with Intel ITT)
-- 
--   Other third-party libraries:
--     Lapack:                      YES (/Library/Developer/CommandLineTools/SDKs/MacOSX12.1.sdk/System/Library/Frameworks/Accelerate.framework -lm -ldl)
--     Eigen:                       YES (ver 3.4.0)
--     Custom HAL:                  YES (carotene (ver 0.0.1))
--     Protobuf:                    build (3.19.1)
-- 
--   OpenCL:                        YES (no extra features)
--     Include path:                NO
--     Link libraries:              -framework OpenCL
-- 
--   Python 2:
--     Interpreter:                 /usr/bin/python2.7 (ver 2.7.18)
--     Libraries:                   /usr/lib/libpython2.7.dylib (ver 2.7.18)
--     numpy:                       /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/core/include (ver 1.8.0rc1)
--     install path:                lib/python2.7/site-packages/cv2/python-2.7
-- 
--   Python (for build):            /usr/bin/python2.7
-- 
--   Java:                          
--     ant:                         NO
--     JNI:                         NO
--     Java wrappers:               NO
--     Java tests:                  NO
-- 
--   Install to:                    /Users/zacky/evision_test/_build/dev/lib/evision/priv
-- -----------------------------------------------------------------
-- 
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5
[  0%] Building C object 3rdparty/ittnotify/CMakeFiles/ittnotify.dir/src/ittnotify/ittnotify_static.c.o
[  0%] Generate opencv4.pc
[  0%] Building C object 3rdparty/zlib/CMakeFiles/zlib.dir/adler32.c.o
[  0%] Building C object 3rdparty/openjpeg/openjp2/CMakeFiles/libopenjp2.dir/thread.c.o
(snip)
[ 72%] Building CXX object modules/dnn/CMakeFiles/opencv_dnn.dir/src/vkcom/vulkan/vk_loader.cpp.o
[ 72%] Linking CXX shared library ../../lib/libopencv_dnn.dylib
[ 72%] Built target opencv_dnn
make[1]: *** [all] Error 2
make: *** [/Users/zacky/evision_test/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 2
could not compile dependency :evision, "mix compile" failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile evision", update it with "mix deps.update evision" or clean it with "mix deps.clean evision"
==> evision_test
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. Try running the
commands "gcc --version" and / or "make --version". If these programs
are not installed, you will be prompted to install them.

    My environment is as follows:

    $ uname -a 
Darwin zackym1air.local 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:29:10 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T8101 arm64
$ elixir -v
Erlang/OTP 24 [erts-12.2.1] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1]

Elixir 1.13.2 (compiled with Erlang/OTP 24)

    Thank you.

    opened by zacky1972 9
  • Evision.DNN.DetectionModel can't setInputParams at YOLOv4

    Evision.DNN.DetectionModel can't setInputParams at YOLOv4

    I tried to use YOLOv4 at Evision.DNN.DetectionModel

    dog = "yolov4/dog.jpg"
weights = "yolov4/yolov4.weights"
config = "yolov4/yolov4.cfg"
classes = "yolov4/coco.names"
mat = Evision.imread(dog)

Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.DetectionModel.detect(mat)

    return error

    {:error,
 "OpenCV(4.6.0) /Users/runner/work/evision/evision/3rd_party/opencv/opencv-4.6.0/modules/dnn/src/model.cpp:98: error: (-201:Incorrect size of input array) Input size not specified in function 'processFrame'\n"}

    i guess need such as python code

    model.setInputParams(1.0, (416, 416), (0.0, 0.0, 0.0), true, false)

    but

    Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams(scale: 1.0, size: {416, 416}, mean: {0, 0, 0}, swapRB: true, crop: false)

    ** (UndefinedFunctionError) function Evision.DNN.Model.setInputParams/2 is undefined or private. Did you mean:

    Evision.DNN.DetectionModel.detectionModel(weights, config: config)
|> Evision.DNN.Model.setInputParams({416, 416})

    {:error, "cannot get `cv::dnn::Model` from `self`: mismatched type or invalid resource?"}

    How about the following solution? 1 Convert the loaded model to Evision.DNN.Model 2 Enable setInputParams in DetectionModel as well

    Evision.DNN.readFromDarknet load and detect are success but, I wan't to impl post process :-(

    Enviroments evision 0.1.16 elixir 1.14.1 elrang 25.0.1 os macOS 12.6

    opened by thehaigo 8
  • small doubt regarding contours in evision

    small doubt regarding contours in evision

    i get the values of findContours after getting is their any method to seperate contours using in built function like in python imutil.grabcontours

    the code is here

    `def grab_contours(cnts):
    # if the length the contours tuple returned by cv2.findContours
    # is 2 then we are using either OpenCV v2.4, v4-beta, or
    # v4-official
    if len(cnts) == 2:
        cnts = cnts[0]

    # if the length of the contours tuple is 3 then we are using
    # either OpenCV v3, v4-pre, or v4-alpha
    elif len(cnts) == 3:
        cnts = cnts[1]

    # otherwise OpenCV has changed their cv2.findContours return
    # signature yet again and I have no idea WTH is going on
    else:
        raise Exception(("Contours tuple must have length 2 or 3, "
            "otherwise OpenCV changed their cv2.findContours return "
            "signature yet again. Refer to OpenCV's documentation "
            "in that case"))

    # return the actual contours array
    return cnts`

    could please help us sir @cocoa-xu @josevalim

    opened by Jagan3534 8
  • An error occur when compiling on Jetson (Linux, aarch64)

    An error occur when compiling on Jetson (Linux, aarch64)

    Hi,

    I compiled evision on NVIDIA Jetson AGX Xavier (Ubuntu 18.04 Linux Kernel 4.9.201-tegra, aarch64), but the following error occurred:

    ==> evision
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   117    0   117    0     0    393      0 --:--:-- --:--:-- --:--:--   393
100 89.8M    0 89.8M    0     0  9401k      0 --:--:--  0:00:09 --:--:-- 14.4M
CMake Error: The source directory "/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/BUILD_OPENEXR=ON" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
Makefile:96: recipe for target '/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt' failed
make: *** [/mnt/nvme/home/zacky/evision/_build/dev/lib/evision/cmake_opencv_4.5.5/modules/python_bindings_generator/headers.txt] Error 1
** (Mix) Could not compile with "make" (exit status: 2).
You need to have gcc and make installed. If you are using
Ubuntu or any other Debian-based system, install the packages
"build-essential". Also install "erlang-dev" package if not
included in your Erlang/OTP version. If you're on Fedora, run
"dnf group install 'Development Tools'".

    Are Linux and aarch64 not supported?

    opened by zacky1972 8
  • imshow function is broken in pre-compiled library.

    imshow function is broken in pre-compiled library.

    I recently updated from elixir 1.0.0-dev to 1.0.9 version of Evision and everything is great till now, just tried getting the image as output and this error popped (Note: All the issue is with cvwait and imshow doesn't work without cvwait).

    IMG_20221011_070312_455

    opened by navrkr 7
  • sorry for what is did . we have a small problem in sort sir

    sorry for what is did . we have a small problem in sort sir

    I apologise if i sound rude, i did not know english that well may be this is why i sounds rude.

    here same we got the error in the elixir in python we wrote like this in elixir we tried to give but in sort 2 nd parameter is doubted sir in Evision hexdoxs we tried to see the example but in new version they dont show contours = sorted(contours, key=cv2.contourArea, reverse=True)

    could you please help us

    opened by Jagan3534 6
  • Can't round trip QRcode encode/decode

    Can't round trip QRcode encode/decode

    QRcode detection is working well (as far as I can test) but when I generate a QRcode and attempt to detect it I am not having any luck. For example:

    iex> qrcode = Evision.QRCodeEncoder.encode Evision.QRCodeEncoder.create(), "This is a string"           
%Evision.Mat{
  channels: 1,
  dims: 2,
  type: {:u, 8},
  raw_type: 0,
  shape: {25, 25},
  ref: #Reference<0.1002129465.3225026580.34308>
}
iex> Evision.QRCodeDetector.detectAndDecode Evision.QRCodeDetector.qrCodeDetector(), qrcode             
{"",
 %Evision.Mat{
   channels: 2,
   dims: 2,
   type: {:f, 32},
   raw_type: 13,
   shape: {1, 4, 2},
   ref: #Reference<0.1002129465.3225026580.34310>
 }, {:error, "empty matrix"}}
    1. The encoded image is very small, {25, 25} and I don't actually know how - even if its possible - to define parameters for the encoder to make the image larger.
    2. I can see there is a module Evision.QRCodeEncoder.Params but I'm not having any success try to define a param structure. For example:
    iex> Evision.QRCodeEncoder.Params.get_mode Evision.QRCodeEncoder.Params.qrcodeencoder_params()
zsh: segmentation fault  iex -S mix

    Thanks for your patience @cocoa-xu while I work out have to navigate the wonderful world of OpenCV/eVision - help and guidance much appreciated.

    opened by kipcole9 6
  • [Feature] opencv-contrib modules

    [Feature] opencv-contrib modules

    I plan to add support for modules in opencv_contrib gradually. Of course, this would bring an inevitable question: not everyone will use modules in opencv_contrib, so including them in the precompiled binaries will increase the footprint.

    Will it be feasible to have an :evision_contrib library in Elixir (like its python counterpart, where we have opencv-python and opencv-python-contrib)?

    Modules (part 1)

    • [x] barcode
    • [x] dnn_superres
    • [x] face
    • [x] img_hash
    • [x] plot
    • [x] quality
    • [x] rapid
    • [x] sfm
    • [x] shape
    • [x] structured_light
    • [x] text
    • [x] tracking
    • [x] xfeatures2d
    • [x] ximgproc
    • [x] xphoto

    Modules (part 2)

    • [x] aruco
    • [x] bgsegm
    • [x] bioinspired
    • [x] hfs
    • [x] line_descriptor
    • [x] mcc
    • [x] reg
    • [x] rgbd
    • [x] saliency
    • [x] stereo
    • [x] surface_matching
    • [x] wechat_qrcode

    Modules (part 3)

    • [x] cudaarithm
    • [x] cudabgsegm
    • [x] cudacodec
    • [x] cudafeatures2d
    • [x] cudafilters
    • [x] cudaimgproc
    • [x] cudalegacy
    • [x] cudaobjdetect
    • [x] cudaoptflow
    • [x] cudastereo
    • [x] cudawarping
    • [x] cudev

    todo

    • [x] Check module name and struct name
    • [ ] Check _Params classes
    • [ ] Examples
    • [ ] Tests
    enhancement 
    opened by cocoa-xu 2
  • [Feature-GUI] Implementing `Evision.Wx`

    [Feature-GUI] Implementing `Evision.Wx`

    As mentioned in #103:

    Even compiling OpenCV with highgui module, sometimes the Evision.HighGui.imshow function does not really work as expected (in evision): the window(s) it opened will not respond to any GUI events until calling Evision.HighGui.waitKey again.

    To make it as good as one would expect it to be, like how it is in C++/Python, perhaps it would require one to invest a few weeks of time for it. But I'm focusing on other aspects of this library right now, also I'm hesitating to do it because I don't know if this would worth that amount time.

    Also, as :wx has official support from Erlang, and it's well tested, and one can even build a relatively complex GUI app with :wx if they want.

    This module will try to make itself easy to use. Meanwhile, the Evision.HighGui module will stay in the source code even after Evision.Wx is completed.

    Here is a non-exhaustive list of desired/prospective functions in Evision.Wx. All of them, including the type and the number of their input arguments, are subject to changes in the future, or will not be implemented if the desired functionality is partially or wholly impossible to achieve in Erlang/Elixir with :wx.

    Functions available in OpenCV's highgui module can be found here.

    Any contributions or suggestions (either about Evision.Wx or Evision.HighGui) are more than welcomed - this also applies to the whole project. :)

    enhancement 
    opened by cocoa-xu 0
  • Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

    Evision (OpenCV's `cv::Mat`) as an Nx.Backend?

    cv::Mat (and its variants) is the OpenCV's implementation of multi-dimensional array, so technically, it can be used as a backend for the Nx (numerical-elixir).

    opened by cocoa-xu 8
  • OpenCV gapi module

    OpenCV gapi module

    Based on the contents of pyopencv_gapi.hpp, it seems that the binding code will need to call an Elixir function and fetch the result.

    This perhaps could be done if we send related ERL_NIF_TERMs to a separate Erlang thread and send the result back. But it's not easy to properly design and write the whole thing.

    Maybe it's not worth the effort... I probably won't port this module anytime soon.

    This module is highly likely to be excluded from evision 0.1.0.

    opened by cocoa-xu 0
Releases(v0.1.25)
Owner
Cocoa
Ph.D student @ UofG
Cocoa
GitHub Repository
Read Japanese manga inside browser with selectable text.

mokuro Read Japanese manga with selectable text inside a browser. See demo: https://kha-white.github.io/manga-demo mokuro_demo.mp4 Demo contains excer

Maciej Budyś 170 Dec 27, 2022
Hand Detection and Finger Detection on Live Feed

Hand-Detection-On-Live-Feed Hand Detection and Finger Detection on Live Feed Getting Started Install the dependencies $ git clone https://github.com/c

Chauhan Mahaveer 2 Jan 02, 2022
Code for CVPR'2022 paper ✨ "Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-Language Model"

PPE ✨ Repository for our CVPR'2022 paper: Predict, Prevent, and Evaluate: Disentangled Text-Driven Image Manipulation Empowered by Pre-Trained Vision-

Zipeng Xu 34 Nov 28, 2022
FOTS Pytorch Implementation

News!!! Recognition branch now is added into model. The whole project has beed optimized and refactored. ICDAR Dataset SynthText 800K Dataset detectio

Ning Lu 599 Dec 19, 2022
A simple Security Camera created using Opencv in Python where images gets saved in realtime in your Dropbox account at every 5 seconds

Security Camera using Opencv & Dropbox This is a simple Security Camera created using Opencv in Python where images gets saved in realtime in your Dro

Arpit Rath 1 Jan 31, 2022
Detect and fix skew in images containing text

Alyn Skew detection and correction in images containing text Image with skew Image after deskew Install and use via pip! Recommended way(using virtual

Kakul 230 Dec 21, 2022
Text Detection from images using OpenCV

EAST Detector for Text Detection OpenCV’s EAST(Efficient and Accurate Scene Text Detection ) text detector is a deep learning model, based on a novel

Abhishek Singh 88 Oct 20, 2022
Extracting Tables from Document Images using a Multi-stage Pipeline for Table Detection and Table Structure Recognition:

Multi-Type-TD-TSR Check it out on Source Code of our Paper: Multi-Type-TD-TSR Extracting Tables from Document Images using a Multi-stage Pipeline for

Pascal Fischer 178 Dec 27, 2022
Learning Camera Localization via Dense Scene Matching, CVPR2021

This repository contains code of our CVPR 2021 paper - "Learning Camera Localization via Dense Scene Matching" by Shitao Tang, Chengzhou Tang, Rui Hua

tangshitao 65 Dec 01, 2022
Shape Detection - It's a shape detection project with OpenCV and Python.

Shape Detection It's a shape detection project with OpenCV and Python. Setup pip install opencv-python for doing AI things. pip install simpleaudio fo

1 Nov 26, 2022
Controlling the computer volume with your hands // OpenCV

HandsControll-AI Controlling the computer volume with your hands // OpenCV Step 1 git clone https://github.com/Hayk-21/HandsControll-AI.git pip instal

Hayk 1 Nov 04, 2021
A toolbox of scene text detection and recognition

FudanOCR This toolbox contains the implementations of the following papers: Scene Text Telescope: Text-Focused Scene Image Super-Resolution [Chen et a

FudanVIC Team 170 Dec 26, 2022
PyNeuro is designed to connect NeuroSky's MindWave EEG device to Python and provide Callback functionality to provide data to your application in real time.

PyNeuro PyNeuro is designed to connect NeuroSky's MindWave EEG device to Python and provide Callback functionality to provide data to your application

Zach Wang 45 Dec 30, 2022
Fatigue Driving Detection Based on Dlib

Fatigue Driving Detection Based on Dlib

5 Dec 14, 2022
OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched

OCRmyPDF adds an OCR text layer to scanned PDF files, allowing them to be searched or copy-pasted. ocrmypdf # it's a scriptable c

jbarlow83 7.9k Jan 03, 2023
Give a solution to recognize MaoYan font.

猫眼字体识别 该 github repo 在于帮助xjtlu的同学们识别猫眼的扭曲字体。已经打包上传至 pypi ，可以使用 pip 直接安装。 猫眼字体的识别不出来的原理与解决思路在采茶上 使用方法： import MaoYanFontRecognize

Aruix 4 Jun 30, 2022
OCR of Chicago 1909 Renumbering Plan

Requirements: Python 3 (probably at least 3.4) pipenv (pip3 install pipenv) tesseract (brew install tesseract, at least if you have a mac and homebrew

ted whalen 2 Nov 21, 2021
The project is an official implementation of our paper "3D Human Pose Estimation with Spatial and Temporal Transformers".

3D Human Pose Estimation with Spatial and Temporal Transformers This repo is the official implementation for 3D Human Pose Estimation with Spatial and

Ce Zheng 363 Dec 28, 2022
Image processing is one of the most common term in computer vision

Image processing is one of the most common term in computer vision. Computer vision is the process by which computers can understand images and videos, and how they are stored, manipulated, and retri

Happy N. Monday 3 Feb 15, 2022
Code for the paper "DewarpNet: Single-Image Document Unwarping With Stacked 3D and 2D Regression Networks" (ICCV '19)

DewarpNet This repository contains the codes for DewarpNet training. Recent Updates [May, 2020] Added evaluation images and an important note about Ma

<a href=[email protected]"> 354 Jan 01, 2023