Measure file similarity in a many-to-many fashion

Last update: Feb 02, 2022

Overview

Mesi

Mesi is a tool to measure the similarity in a many-to-many fashion of long-form documents like Python source code or technical writing. The output can be useful in determining which of a collection of files are the most similar to each other.

Installation

Python 3.9+ and pipx are recommended, although Python 3.6+ and/or pip will also work.

pipx install mesi

If you'd like to test out Mesi before installing it, use the remote execution feature of pipx, which will temporarily download Mesi and run it in an isolated virtual environment.

pipx run mesi --help

Usage

For a directory structure that looks like:

lab-one
├── StudentOne
│   ├── pyproject.toml
│   ├── deliverables
│   │   └── python_program.py
│   └── README.md
├── StudentTwo
│   ├── pyproject.toml
│   ├── deliverables
│   │   └── python_program.py
│   └── README.md
│

where similarity should be measured between each student's deliverables/python_program.py file, run the command:

mesi lab-one/*/deliverables/python_program.py

A lower distance in the produced table equates to a higher degree of similarity.

See the help menu (mesi --help) for additional options and configuration.

Algorithms

There are many algorithms to choose from when comparing string similarity! Mesi implements all the algorithms provided by TextDistance. In general levenshtein is never a bad choice, which is why it is the default.

Bugs/Requests

Please use the GitHub issue tracker to submit bugs or request new features, options, or algorithms.

Dependencies

Mesi uses two primary dependencies for text similarity calculation: polyleven, and TextDistance. Polyleven is the default, as its singular implementation of Levenshtein distance can be faster in most situations. However, if a different edit distance algorithm is requested, TextDistance's implementations will be used.

License

Distributed under the terms of the GPL v3 license, mesi is free and open source software.

File-manager - A basic file manager, written in Python

File Manager A basic file manager, written in Python. Installation Install Pytho

1 Feb 5, 2022

Two scripts help you to convert csv file to md file by template

Two scripts help you to convert csv file to md file by template. One help you generate multiple md files with different filenames from the first colume of csv file. Another can generate one md file with several blocks.

2 Oct 15, 2022

A simple Python code that takes input from a csv file and makes it into a vcf file.

Contacts-Maker A simple Python code that takes input from a csv file and makes it into a vcf file. Imagine a college or a large community where each y

1 Feb 13, 2022

This program can help you to move and rename many files at once

This program can help you to rename and save many files in a folder in seconds, but don't give the same name to files, it can delete both files.

1 Oct 10, 2022

Object-oriented file system path manipulation

path (aka path pie, formerly path.py) implements path objects as first-class entities, allowing common operations on files to be invoked on those path

1k Dec 28, 2022

An object-oriented approach to Python file/directory operations.

Unipath An object-oriented approach to file/directory operations Version: 1.1 Home page: https://github.com/mikeorr/Unipath Docs: https://github.com/m

506 Dec 29, 2022

File support for asyncio

aiofiles: file support for asyncio aiofiles is an Apache2 licensed library, written in Python, for handling local disk files in asyncio applications.

2.1k Jan 1, 2023

Object-oriented file system path manipulation

path (aka path pie, formerly path.py) implements path objects as first-class entities, allowing common operations on files to be invoked on those path

1k Dec 28, 2022

A platform independent file lock for Python

py-filelock This package contains a single module, which implements a platform independent file lock in Python, which provides a simple way of inter-p

497 Jan 5, 2023

Releases(v1.1.0)

v1.1.0(Dec 8, 2021)

This release adds an --average option, which prints the average distance computed against the given files after the individual comparison distances. An unimplemented --distribution option, intended for use in the future, was also added.
Source code(tar.gz)
Source code(zip)
v1.0.2(Oct 26, 2021)

This release adds more documentation, a --version option, and removes the bright white table styling from previous versions that were hard to read on light-themed terminals.

Full Changelog: https://github.com/Michionlion/mesi/compare/v1.0.1...v1.0.2
Source code(tar.gz)
Source code(zip)
v1.0.1(Oct 26, 2021)

This release fixes a bug where in some situations, some distinct parts of compared file names were not displayed. Additionally, a new option, --table-format, was added. This allows configuration of the output table format, using tabulate's table formats.

Full Changelog: https://github.com/Michionlion/mesi/compare/v1.0.0...v1.0.1
Source code(tar.gz)
Source code(zip)
v1.0.0(Oct 25, 2021)
Mesi is a tool to check the similarity between many different files; this is its first release, with support for many basic features and lots of text similarity/distance algorithms. You can even try out Mesi without downloading it!

pipx run mesi --help

If you don't have pipx installed, you may need to install it, but that's something you should do anyways. :smile:
Source code(tar.gz)
Source code(zip)

Owner

GatorEducator

Software tools developed at Allegheny College for computer science courses

GitHub Repository

A Certificate renaming tool made for IEEE CS SBC, SJCE.

PDF Batch Renamer Made for IEEE CS SBC, SJCE How to use? Before using the python script, ensure that pytesseract, pdf2image, opencv and other supporti

2 Nov 14, 2021

Maltego transforms to pivot between PE files based on their VirusTotal codeblocks

VirusTotal Codeblocks Maltego Transforms Introduction These Maltego transforms allow you to pivot between different PE files based on codeblocks they

18 Feb 03, 2022

csv2ir is a script to convert ir .csv files to .ir files for the flipper.

csv2ir csv2ir is a script to convert ir .csv files to .ir files for the flipper. For a repo of .ir files, please see https://github.com/logickworkshop

38 Dec 31, 2022

Convert All TXT Files To One File.

AllToOne Convert All TXT Files To One File. Hi 👋 , I'm Alireza A Python Developer Boy 🔭 I’m currently working on my C# projects 🌱 I’m currently Lea

4 Jun 07, 2022

Copy only text-like files from the folder

copy-only-text-like-files-from-folder-python copy only text-like files from the folder This project is for those who want to copy only source code or

1 May 17, 2022

Better directory iterator and faster os.walk(), now in the Python 3.5 stdlib

scandir, a better directory iterator and faster os.walk() scandir() is a directory iteration function like os.listdir(), except that instead of return

506 Dec 29, 2022

This python project contains a class FileProcessor which allows one to grab a file and get some meta data and header information from it

This python project contains a class FileProcessor which allows one to grab a file and get some meta data and header information from it. In the current state, it outputs a PrettyTable to txt file as

1 Nov 09, 2021

The best way to convert files on your computer, be it .pdf to .png, .pdf to .docx, .png to .ico, or anything you can imagine.

2 Nov 20, 2021

PaddingZip - a tool that you can craft a zip file that contains the padding characters between the file content.

53 Nov 07, 2022

A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".

the problem What directory should your app use for storing user data? If running on macOS, you should use: ~/Library/Application Support/AppName If

948 Dec 31, 2022

Annotate your Python requirements.txt file with summaries of each package.

Summarize Requirements 🐍 📜 Annotate your Python requirements.txt file with a short summary of each package. This tool: takes a Python requirements.t

8 Apr 22, 2022

Python function to stream unzip all the files in a ZIP archive: without loading the entire ZIP file or any of its files into memory at once

206 Jan 02, 2023

Measure file similarity in a many-to-many fashion

Related tags

Overview

Mesi

Installation

Usage

Algorithms

Bugs/Requests

Dependencies

License

You might also like...

File-manager - A basic file manager, written in Python

Two scripts help you to convert csv file to md file by template

A simple Python code that takes input from a csv file and makes it into a vcf file.

This program can help you to move and rename many files at once

Object-oriented file system path manipulation

An object-oriented approach to Python file/directory operations.

File support for asyncio

Object-oriented file system path manipulation

A platform independent file lock for Python

Releases(v1.1.0)

v1.1.0(Dec 8, 2021)

v1.0.2(Oct 26, 2021)

v1.0.1(Oct 26, 2021)

v1.0.0(Oct 25, 2021)

Owner

GatorEducator

A Certificate renaming tool made for IEEE CS SBC, SJCE.

Maltego transforms to pivot between PE files based on their VirusTotal codeblocks

csv2ir is a script to convert ir .csv files to .ir files for the flipper.

Convert All TXT Files To One File.

Copy only text-like files from the folder

Better directory iterator and faster os.walk(), now in the Python 3.5 stdlib

This python project contains a class FileProcessor which allows one to grab a file and get some meta data and header information from it

The best way to convert files on your computer, be it .pdf to .png, .pdf to .docx, .png to .ico, or anything you can imagine.

PaddingZip - a tool that you can craft a zip file that contains the padding characters between the file content.

A small Python module for determining appropriate platform-specific dirs, e.g. a "user data dir".

Annotate your Python requirements.txt file with summaries of each package.

Python function to stream unzip all the files in a ZIP archive: without loading the entire ZIP file or any of its files into memory at once

Various converters to convert value sets from CSV to JSON, etc.

Yadl - it is a simple library for working with both dotenv files and environment variables.

Fast Python reader and editor for ASAM MDF / MF4 (Measurement Data Format) files

Import Python modules from any file system path

pytiff is a lightweight library for reading chunks from a tiff file

Python function to construct a ZIP archive with on the fly - without having to store the entire ZIP in memory or disk

A python module to parse text files with contains secret variables.

Sheet Data Image/PDF-to-CSV Converter