Lingtrain Studio is the ML based app for accurate texts alignment on different languages.

• Extracts parallel corpora from two texts.
• Makes the formatted parallel book from it with sentence highlighting.

• How to create bilingual books. Part 2. Lingtrain Alignment Studio
• How to make a parallel texts for language learning. Part 1. Python and Colab version
• Lingtrain Aligner. Приложение для создания параллельных книг, которое вас удивит
• Сам себе Гутенберг. Делаем параллельные книги

Automated alignment process relies on the sentence embeddings models. Embeddings are multidimensional vectors of a special kind which are used to calculate a distance between the sentences. You can also plug your own model using the interface described in models directory. Supported languages list depends on the selected backend model.

• distiluse-base-multilingual-cased-v2
  • more reliable and fast
  • moderate weights size — 500MB
  • supports 50+ languages
  • full list of supported languages can be found in this paper
• LaBSE (Language-agnostic BERT Sentence Embedding)
  • can be used for rare languages
  • pretty heavy weights — 1.8GB
  • supports 100+ languages
  • full list of supported languages can be found here

You can run the application on your computer using docker.

1. Make sure that docker is installed by typing the docker version command in your console.

2. Images configured to run locally are available on Docker Hub.

3. Run the following commands in your console:

   • docker pull lingtrain/studio:v7.2
   • docker run -v C:\app\data:/app/data -v C:\app\img:/app/static/img -p 80:80 lingtrain/studio:v7.2

4. App will be available in your browser on the localhost address.

5. If you need to run the container on another port (e.g. localhost:8081):

   • Change the API_URL parameter in config.js
   • Rebuild the docker container
   • Start it with changed -p parameter (e.g. -p 8081:80)

Clone this repo on your machine.

Flask/uwsgi backend REST API service. It contains all the alignment logic.

• Go to the backend directory

  • cd /backend

• Install the requirements

  • pip install -r requirements.txt

• Run the backend application

  • python main.py

SPA. Vue + vuex + vuetify. UI for managing alignment process using BE and a tool for translators to edit processing documents.

• Go to the frontend directory

  • cd /frontend

• Install the requirements

  • npm install -f

• Compile and run with hot-reloads for development

  • npm run serve

Application will be available on localhost:8080

You can create an issue or send me a message in telegram: @averkij

This work is licensed under a Attribution-NonCommercial-NoDerivatives 4.0 International license. See LICENSE.