Official implementation of the MM'21 paper Constrained Graphic Layout Generation via Latent Optimization

Overview

[MM'21] Constrained Graphic Layout Generation via Latent Optimization

This repository provides the official code for the paper "Constrained Graphic Layout Generation via Latent Optimization", especially the code for:

  • LayoutGAN++: generative adversarial networks for layout generation
  • CLG-LO: a framework for generating layouts that satisfy constraints
  • Layout evaluation: measuring the quantitative metrics of Layout FID, Maximum IoU, Alignment, and Overlap for generated layouts

Installation

  1. Clone this repository

    git clone https://github.com/ktrk115/const_layout.git
    cd const_layout
  2. Create a new conda environment (Python 3.8)

    conda create -n const_layout python=3.8
    conda activate const_layout
  3. Install PyTorch 1.8.* and the corresponding versoin of PyTorch Geometric

  4. Install the other dependent libraries

    pip install -r requirements.txt
  5. Prepare data (see this instruction)

  6. Download pre-trained models

    ./download_model.sh

Development environment

  • Ubuntu 18.04, CUDA 11.1

LayoutGAN++

Architecture

Training animation

Generate layouts with LayoutGAN++

python generate.py pretrained/layoutganpp_rico.pth.tar --out_path output/generated_layouts.pkl --num_save 5

Train LayoutGAN++ model

python train.py --dataset rico --batch_size 64 --iteration 200000 --latent_size 4 --lr 1e-05 --G_d_model 256 --G_nhead 4 --G_num_layers 8 --D_d_model 256 --D_nhead 4 --D_num_layers 8

CLG-LO

w/ beautification constraints w/ relational constraints

Generate layouts with beautification constraints

python generate_const.py pretrained/layoutganpp_publaynet.pth.tar --const_type beautify --out_path output/beautify/generated_layouts.pkl --num_save 5

Generate layouts with relational constraints

python generate_const.py pretrained/layoutganpp_publaynet.pth.tar --const_type relation --out_path output/relation/generated_layouts.pkl --num_save 5

Layout evaluation

Evaluate generated layouts

python eval.py rico output/generated_layouts.pkl

A pickle file should be a list of layouts, where each layout is a tuple of bounding boxes and labels. The bounding box is represented by [x, y, width, height] in normalized coordinates, and the label is represented by an index. An example is shown below.

In [x]: layouts
Out[x]:
[(array([[0.47403812, 0.11276676, 0.6250037 , 0.02210438],
         [0.49971417, 0.8550553 , 0.81388366, 0.03492427],
         [0.49919674, 0.47857162, 0.81024694, 0.7070079 ]], dtype=float32),
  array([0, 0, 3]),
  ...

Citation

If this repository helps your research, please consider citing our paper.

@inproceedings{Kikuchi2021,
    title = {Constrained Graphic Layout Generation via Latent Optimization},
    author = {Kotaro Kikuchi and Edgar Simo-Serra and Mayu Otani and Kota Yamaguchi},
    booktitle = {Proceedings of the ACM International Conference on Multimedia},
    series = {MM '21},
    volume = {},
    year = {2021},
    pages = {},
    doi = {10.1145/3474085.3475497}
}

Licence

GNU AGPLv3

Related repositories

Owner
Kotaro Kikuchi
Waseda University
Kotaro Kikuchi
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search

BossNAS This repository contains PyTorch evaluation code, retraining code and pretrained models of our paper: BossNAS: Exploring Hybrid CNN-transforme

Changlin Li 127 Dec 26, 2022
Everything's Talkin': Pareidolia Face Reenactment (CVPR2021)

Everything's Talkin': Pareidolia Face Reenactment (CVPR2021) Linsen Song, Wayne Wu, Chaoyou Fu, Chen Qian, Chen Change Loy, and Ran He [Paper], [Video

71 Dec 21, 2022
Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

Group-CAM By Zhang, Qinglong and Rao, Lu and Yang, Yubin [State Key Laboratory for Novel Software Technology at Nanjing University] This repo is the o

zhql 98 Nov 16, 2022
A library for differentiable nonlinear optimization.

Theseus A library for differentiable nonlinear optimization built on PyTorch to support constructing various problems in robotics and vision as end-to

Meta Research 1.1k Dec 30, 2022
Implementation of Heterogeneous Graph Attention Network

HetGAN Implementation of Heterogeneous Graph Attention Network This is the code repository of paper "Prediction of Metro Ridership During the COVID-19

5 Dec 28, 2021
A simple rest api serving a deep learning model that classifies human gender based on their faces. (vgg16 transfare learning)

this is a simple rest api serving a deep learning model that classifies human gender based on their faces. (vgg16 transfare learning)

crispengari 5 Dec 09, 2021
This is the latest version of the PULP SDK

PULP-SDK This is the latest version of the PULP SDK, which is under active development. The previous (now legacy) version, which is no longer supporte

78 Dec 07, 2022
Training a Resilient Q-Network against Observational Interference, Causal Inference Q-Networks

Obs-Causal-Q-Network AAAI 2022 - Training a Resilient Q-Network against Observational Interference Preprint | Slides | Colab Demo | Environment Setup

23 Nov 21, 2022
Pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering".

TRAnsformer Routing Networks (TRAR) This is an official implementation for ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visu

Ren Tianhe 49 Nov 10, 2022
PyGAD, a Python 3 library for building the genetic algorithm and training machine learning algorithms (Keras & PyTorch).

PyGAD: Genetic Algorithm in Python PyGAD is an open-source easy-to-use Python 3 library for building the genetic algorithm and optimizing machine lear

Ahmed Gad 1.1k Dec 26, 2022
Plaything for Autistic Children (demo for PaddlePaddle/Wechaty/Mixlab project)

星星的孩子 - 一款为孤独症孩子设计的聊天机器人游戏 孤独症儿童是目前常常被忽视的一类群体。他们有着类似性格内向的特征,实际却受着广泛性发育障碍的折磨。 项目背景 这类儿童在与人交往时存在着沟通障碍,其特点表现在: 社交交流差,互动障碍明显 认知能力有限,被动认知 兴趣狭窄,重复刻板,缺乏变化和想象

Tianyi Pan 35 Nov 24, 2022
This is the pytorch code for the paper Curious Representation Learning for Embodied Intelligence.

Curious Representation Learning for Embodied Intelligence This is the pytorch code for the paper Curious Representation Learning for Embodied Intellig

19 Oct 19, 2022
Doing the asl sign language classification on static images using graph neural networks.

SignLangGNN When GNNs 💜 MediaPipe. This is a starter project where I tried to implement some traditional image classification problem i.e. the ASL si

10 Nov 09, 2022
This repository contains the code for: RerrFact model for SciVer shared task

RerrFact This repository contains the code for: RerrFact model for SciVer shared task. Setup for Inference 1. Download SciFact database Download the S

Ashish Rana 1 May 22, 2022
'A C2C E-COMMERCE TRUST MODEL BASED ON REPUTATION' Python implementation

Project description A library providing functionalities to calculate reputation and degree of trust on C2C ecommerce platforms. The work is fully base

Davide Bigotti 2 Dec 14, 2022
Time should be taken seer-iously

TimeSeers seers - (Noun) plural form of seer - A person who foretells future events by or as if by supernatural means TimeSeers is an hierarchical Bay

279 Dec 26, 2022
House_prices_kaggle - Predict sales prices and practice feature engineering, RFs, and gradient boosting

House Prices - Advanced Regression Techniques Predicting House Prices with Machine Learning This project is build to enhance my knowledge about machin

Gurpreet Singh 1 Jan 01, 2022
On-device speech-to-intent engine powered by deep learning

Rhino Made in Vancouver, Canada by Picovoice Rhino is Picovoice's Speech-to-Intent engine. It directly infers intent from spoken commands within a giv

Picovoice 510 Dec 30, 2022
Patches desktop steam to look like the new steamdeck ui.

steam_deck_ui_patch The Deck UI patch will patch the regular desktop steam to look like the brand new SteamDeck UI. This patch tool currently works on

The_IT_Dude 3 Aug 29, 2022
A small library for doing fluid simulation with neural networks.

Neural Fluid Fields This is a small library for doing fluid simulation with neural fields. Check out our review paper, Neural Fields in Visual Computi

Towaki 23 Jun 23, 2022