Official implementation of SynthTIGER (Synthetic Text Image GEneratoR) ICDAR 2021

Overview

đŸ¯ SynthTIGER: Synthetic Text Image GEneratoR

Official implementation of SynthTIGER | Paper | Datasets

Moonbin Yim1, Yoonsik Kim1, Han-cheol Cho1, Sungrae Park2

1 Clova AI Research, NAVER Corp.

2 Upstage AI Research





Contents

Updates

Datasets

SynthTIGER is available for download at google drive.

synthtiger_v1.0.zip (36G) (md5: 5b5365f4fe15de24e403a9256079be70)

  • Original paper version.

synthtiger_v1.1.zip (38G) (md5: b2757a7e2b5040b14ed64c473533b592)

  • Used MJ/ST lexicon instead of MJ/ST label.
  • Fixed a bug that applies transformation twice on curved text.
  • Fixed a bug that incorrectly converts grayscale to RGB.
Version IIIT5k SVT IC03 IC13 IC15 SVTP CUTE80 Total
1.0 93.2 87.3 90.5 92.9 72.1 77.7 80.6 85.9
1.1 93.4 87.6 91.4 93.2 73.9 77.8 80.6 86.6

Structure

The structure of the dataset is as follows. The dataset contains 10M images.

gt.txt
images/
    0/
        0.jpg
        1.jpg
        ...
        9998.jpg
        9999.jpg
    1/
    ...
    998/
    999/

The format of gt.txt is as follows. Image path and label are separated by tab. (<image_path>\t<label>)

images/0/0.jpg	10
images/0/1.jpg	date:
...
images/999/9999998.jpg	STUFFIER
images/999/9999999.jpg	Re:

Usage

# for macOS
$ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

# install python packages
$ pip install -r requirements.txt

$ python gen.py --template TEMPLATE
                --config CONFIG
                --output OUTPUT
                [--count COUNT]
                [--worker WORKER]

Requirements

  • python >= 3.6
  • libraqm

Parameters

Name Type Default Description
template string Template module path
config string Config file path
output string Folder path to save data
count integer 100 Number of data
worker integer 1 Number of workers

Examples

Default text images

# horizontal
python gen.py --template templates/default.py --config templates/default_horizontal.yaml --output results --worker 4

# vertical
python gen.py --template templates/default.py --config templates/default_vertical.yaml --output results --worker 4

Multiline text images

python gen.py --template templates/multiline.py --config templates/multiline.yaml --output results --worker 4

Advanced Usage

Non-Latin language data generation

  1. Prepare corpus and fonts

    corpus - txt file, line by line (example)

    font - ttf/otf file (example)

  2. Extract renderable charsets

    python tools/extract_font_charset.py --input fonts --worker 4

    This script extracts renderable charsets for all font files. (example)

    Text files are generated in the input path with the same names as the fonts.

  3. Edit corpus path and font path in config file

  4. Run gen.py

Colormap customization

  1. Prepare images

    image - jpg/jpeg/png/bmp file

  2. Create colormaps

    python tools/create_colormap.py --input images --output colormap.txt --worker 4

    This script creates colormaps for all image files. (example)

  3. Edit colormap path in config file

  4. Run gen.py

Citation

@article{yim2021synthtiger,
  title={SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Models},
  author={Yim, Moonbin and Kim, Yoonsik and Cho, Han-Cheol and Park, Sungrae},
  journal={arXiv preprint arXiv:2107.09313},
  year={2021}
}

License

SynthTIGER
Copyright (c) 2021-present NAVER Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

The following directories and their subdirectories are licensed the same as their origins. Please refer to NOTICE

docs/
docsrc/
resources/font/
Comments
  • How to get character bbox annotation?

    How to get character bbox annotation?

    this project is very helpful for generating synth text for scene text recognition, and it seem to generate text image by combine several character images, but the outputs doesn't contain information of each character, is it possible to get character annotation, for example, each character and its location?

    opened by GuokunWang 5
  • Question about background image

    Question about background image

    Hi, @moonbings Thank you for sharing nice work.

    I wonder where's the code that load background image such as below?

    image

    And I also wonder when the Synthdog will be released

    opened by yellowjs0304 3
  • training configuration of STR model

    training configuration of STR model

    Hi, thank you for open-sourcing your work. I have a question about the training configuration of STR model used in your paper. How did you set the sensitive character mode and data_filtering_off option in BEST model?

    opened by tzm-tora 2
  • gen.py throws munmap_chunk(): invalid pointer and stops generating images

    gen.py throws munmap_chunk(): invalid pointer and stops generating images

    On running gen.py with the command posted in the readme, the process throws

    munmap_chunk(): invalid pointer

    and is stuck. It stops generating images. Each worker generates a maximum of 1 image before this error is thrown.

    opened by arundprabhu 2
  • libraqm dependency error occurs

    libraqm dependency error occurs

    Thanks for the nice package! When i followed your descriptions and used it, i met some error. KeyError: 'setting text direction, language or font features is not supported without libraqm' I have installed python package 'synthtiger' and dependency following the shell script How can i solve this issue? Thanks,

    opened by mandal4 1
  • How to generate Non-Broken bengali synthetic data for text recognition?

    How to generate Non-Broken bengali synthetic data for text recognition?

    thank you for your awesome sharing. i tried to generate text recognition data using your library for bangla printed documents,i made all the necessary changes to make this synthesis engine working for bengali text recognition data generation, however for bangla it is breaking text during text to image conversion just like this trdg issue of mine : https://github.com/Belval/TextRecognitionDataGenerator/issues/253 (where the authors of that repo is not responsive at all)

    here is a sample that i got while using this synthesis engine.

    label : āύāĻŋāĻ°ā§āϞāĻœā§āϜ

    image : https://i.ibb.co/0Y0W5GG/image.jpg ( āύāĻŋāĻ°ā§āϞāĻœā§āϜ word got broken )

    do you know how to solve this issue? thanks a lot in advance for your great work.

    opened by mobassir94 1
  • Other latin language

    Other latin language

    Hi, great work ! What exactly should one modify to change language. I found this: https://github.com/clovaai/synthtiger/tree/master/resources/charset but not sure if I need to generate something from that.

    thanks in advance :)

    opened by Globolik 1
  • Fix seed bug and update test code

    Fix seed bug and update test code

    Description

    • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
    • Added retry parameter in generator function to make testing easier.
      • If retry is False, when an error occurs during data generation, it does not retry and return data of None value.
    • Updated test code.

    Changes in this PR

    • Updated main.py, gen.py.
    • Updated test code.

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update seed feature

    Update seed feature

    Description

    • Random seed is also applied to main process.
    • Added get_global_random_states, set_global_random_states, set_global_random_seed functions. This functions can be used in templates.

    Changes in this PR

    • Updated main.py, gen.py.
    • Added functions (get_global_random_states, set_global_random_states, set_global_random_seed).

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Add seed option

    Add seed option

    Description

    This engine cannot reproduce same data. So I added random seed option to solve this issue.

    I also changed multi processing structure. When the new engine puts tasks in the task queue, each worker gets and runs it. And each task is assigned an index. Note that the incoming indexes are random when storing data because the task completion times are different.

    Changes in this PR

    • Added random seed option. (-s or --seed)
    • Refactored main.py, gen.py
    • Updated README.md
    • Updated .pylintrc

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update ubuntu depends

    Update ubuntu depends

    Description

    Updated shell script to get latest package list before installing dependencies in ubuntu.

    Changes in this PR

    • Updated install_ubuntu_depends.sh.

    How has this been tested?

    Check CI test.

    opened by moonbings 0
  • Issue with synthtiger_v1.1.zip

    Issue with synthtiger_v1.1.zip

    Hi All,

    I have downloaded all parts of synthtiger_v1.1.zip. When I combined into a single zip file, the resulting zip file is not valid.

    Anyone faces similar problem ?

    Appreciate your advices.

    opened by rinabuoy 1
  • Improving documentation

    Improving documentation

    Hi team,

    Is there any chance you could improve the documentation for this library to describe what the functions and classes actually do? I'm having trouble figuring out how to utilize this library for different kinds of unstructured & semi-structured text images.

    opened by haydenedelson 0
  • Images are very distorted

    Images are very distorted

    Hello @moonbings During synthetic dataset generation, some image are very distorted and i don't have any idea how to fix it. I played with some parameters but it didnt work for me. Any solution?

    Samples image are not clear 23 33 83

    opened by khawar-islam 1
  • RuntimeError: Text is not visible

    RuntimeError: Text is not visible

    Hello @moonbings While generating Korean dataset, I have added all given requirements like fonts and everything. I am using below command to generate dataset but getting an error between data generation process.

    python -m synthtiger -o results -w 4 -v examples/synthtiger/template.py SynthTiger examples/synthtiger/config_vertical.yaml

    Generated 56 data
    Generated 57 data
    Generated 58 data
    Generated 59 data
    Generated 60 data
    Traceback (most recent call last):
      File "/media/cvpr/CM_22/synthtiger/synthtiger/gen.py", line 71, in _generate
        data = template.generate()
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 117, in generate
        image = _blend_images(fg_image, bg_image, self.visibility_check)
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 246, in _blend_images
        raise RuntimeError("Text is not visible")
    RuntimeError: Text is not visible
    
    Generated 61 data
    Generated 62 data
    Generated 63 data
    Generated 64 data
    Generated 65 data
    Generated 66 data
    
    opened by khawar-islam 1
Releases(1.2.1)
  • 1.2.1(Nov 11, 2022)

    • Fix seed bug and update test code (https://github.com/clovaai/synthtiger/pull/46)
      • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
      • Added retry parameter in generator function to make testing easier.
      • Updated test code.
    Source code(tar.gz)
    Source code(zip)
  • 1.2.0(Nov 10, 2022)

    • Updated seed feature (https://github.com/clovaai/synthtiger/pull/44)
      • Random seed is also applied to main process.
      • Added get_global_random_states, set_global_random_states, set_global_random_seed functions.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Nov 9, 2022)

    • Added character bbox and mask outputs (https://github.com/clovaai/synthtiger/pull/33, https://github.com/clovaai/synthtiger/pull/39)
    • Added seed option (https://github.com/clovaai/synthtiger/pull/42)
    • Changed multi processing structure (https://github.com/clovaai/synthtiger/pull/42)
      • The incoming indexes are random when storing data because the task completion times are different.
    • Updated ubuntu depends (https://github.com/clovaai/synthtiger/pull/41)
    • Updated some style and add some information in README.md (https://github.com/clovaai/synthtiger/pull/40)
    Source code(tar.gz)
    Source code(zip)
  • 1.0.2(May 8, 2022)

  • 1.0.1(Feb 17, 2022)

  • 1.0.0(Feb 15, 2022)

Owner
Clova AI Research
Open source repository of Clova AI Research, NAVER & LINE
Clova AI Research
Release of the ConditionalQA dataset

ConditionalQA Datasets accompanying the paper ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers. Disclaimer This dataset

14 Oct 17, 2022
Centroid-UNet is deep neural network model to detect centroids from satellite images.

Centroid UNet - Locating Object Centroids in Aerial/Serial Images Introduction Centroid-UNet is deep neural network model to detect centroids from Aer

GIC-AIT 19 Dec 08, 2022
Learned image compression

Overview Pytorch code of our recent work A Unified End-to-End Framework for Efficient Deep Image Compression. We first release the code for Variationa

Jiaheng Liu 163 Dec 04, 2022
Pytorch implementation of the AAAI 2022 paper "Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification"

[AAAI22] Cross-Domain Empirical Risk Minimization for Unbiased Long-tailed Classification We point out the overlooked unbiasedness in long-tailed clas

PatatiPatata 28 Oct 18, 2022
RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation (CIKM'17)

RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation This is the implementation of RATE: Overcoming Noise and Spar

Yu Zhang 5 Feb 10, 2022
MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python

MNE-Python MNE-Python software is an open-source Python package for exploring, visualizing, and analyzing human neurophysiological data such as MEG, E

MNE tools for MEG and EEG data analysis 2.1k Dec 28, 2022
Unofficial JAX implementations of Deep Learning models

JAX Models Table of Contents About The Project Getting Started Prerequisites Installation Usage Contributing License Contact About The Project The JAX

107 Jan 05, 2023
This code provides various models combining dilated convolutions with residual networks

Overview This code provides various models combining dilated convolutions with residual networks. Our models can achieve better performance with less

Fisher Yu 1.1k Dec 30, 2022
Code for the tech report Toward Training at ImageNet Scale with Differential Privacy

Differentially private Imagenet training Code for the tech report Toward Training at ImageNet Scale with Differential Privacy by Alexey Kurakin, Steve

Google Research 29 Nov 03, 2022
This is RFA-Toolbox, a simple and easy-to-use library that allows you to optimize your neural network architectures using receptive field analysis (RFA) and create graph visualizations of your architecture.

ReceptiveFieldAnalysisToolbox This is RFA-Toolbox, a simple and easy-to-use library that allows you to optimize your neural network architectures usin

84 Nov 23, 2022
MAVE: : A Product Dataset for Multi-source Attribute Value Extraction

MAVE: : A Product Dataset for Multi-source Attribute Value Extraction The dataset contains 3 million attribute-value annotations across 1257 unique ca

Google Research Datasets 89 Jan 08, 2023
NAS-Bench-x11 and the Power of Learning Curves

NAS-Bench-x11 NAS-Bench-x11 and the Power of Learning Curves Shen Yan, Colin White, Yash Savani, Frank Hutter. NeurIPS 2021. Surrogate NAS benchmarks

AutoML-Freiburg-Hannover 13 Nov 18, 2022
Implementation of "Fast and Flexible Temporal Point Processes with Triangular Maps" (Oral @ NeurIPS 2020)

Fast and Flexible Temporal Point Processes with Triangular Maps This repository includes a reference implementation of the algorithms described in "Fa

Oleksandr Shchur 20 Dec 02, 2022
Text to Image Generation with Semantic-Spatial Aware GAN

text2image This repository includes the implementation for Text to Image Generation with Semantic-Spatial Aware GAN This repo is not completely. Netwo

CVDDL 124 Dec 30, 2022
Deploying PyTorch Model to Production with FastAPI in CUDA-supported Docker

Deploying PyTorch Model to Production with FastAPI in CUDA-supported Docker A example FastAPI PyTorch Model deploy with nvidia/cuda base docker. Model

Ming 68 Jan 04, 2023
Hi Guys, here I am providing examples, which will help you in Lerarning Python

LearningPython Hi guys, here I am trying to include as many practice examples of Python Language, as i Myself learn, and hope these will help you in t

4 Feb 03, 2022
Implementation of the famous Image Manipulation\Forgery Detector "ManTraNet" in Pytorch

Who has never met a forged picture on the web ? No one ! Everyday we are constantly facing fake pictures touched up in Photoshop but it is not always

Rony Abecidan 77 Dec 16, 2022
Notspot robot simulation - Python version

Notspot robot simulation - Python version This repository contains all the files and code needed to simulate the notspot quadrupedal robot using Gazeb

50 Sep 26, 2022
Source code for "Understanding Knowledge Integration in Language Models with Graph Convolutions"

Graph Convolution Simulator (GCS) Source code for "Understanding Knowledge Integration in Language Models with Graph Convolutions" Requirements: PyTor

yifan 10 Oct 18, 2022
Reference code for the paper "Cross-Camera Convolutional Color Constancy" (ICCV 2021)

Cross-Camera Convolutional Color Constancy, ICCV 2021 (Oral) Mahmoud Afifi1,2, Jonathan T. Barron2, Chloe LeGendre2, Yun-Ta Tsai2, and Francois Bleibe

Mahmoud Afifi 76 Jan 07, 2023