Python flexible slugify function

Overview

awesome-slugify

https://travis-ci.org/dimka665/awesome-slugify.svg?branch=master

Python flexible slugify function

Install

pip install awesome-slugify

Usage

from slugify import slugify

slugify('Any text')  # 'Any-text'

Custom slugify

from slugify import slugify, Slugify, UniqueSlugify

slugify('Any text', to_lower=True)  # 'any-text'

custom_slugify = Slugify(to_lower=True)
custom_slugify('Any text')          # 'any-text'

custom_slugify.separator = '_'
custom_slugify('Any text')          # 'any_text'

custom_slugify = UniqueSlugify()
custom_slugify('Any text')          # 'any-text'
custom_slugify('Any text')          # 'any-text-1'

slugify function optional args

to_lower              # if True convert text to lowercase
max_length            # output string max length
separator             # separator string
capitalize            # if True upper first letter

Slugify class args

pretranslate = None               # function or dict for replace before translation
translate = unidecode.unidecode   # function for slugifying or None
safe_chars = ''                   # additional safe chars
stop_words = ()                   # remove these words from slug

to_lower = False                  # default to_lower value
max_length = None                 # default max_length value
separator = '-'                   # default separator value
capitalize = False                # default capitalize value

UniqueSlugify class args

# all Slugify class args +
uids = []                         # initial unique ids

Predefined slugify functions

Some slugify functions is predefined this way:

from slugify import Slugify, CYRILLIC, GERMAN, GREEK

slugify = Slugify()
slugify_unicode = Slugify(translate=None)

slugify_url = Slugify()
slugify_url.to_lower = True
slugify_url.stop_words = ('a', 'an', 'the')
slugify_url.max_length = 200

slugify_filename = Slugify()
slugify_filename.separator = '_'
slugify_filename.safe_chars = '-.'
slugify_filename.max_length = 255

slugify_ru = Slugify(pretranslate=CYRILLIC)
slugify_de = Slugify(pretranslate=GERMAN)
slugify_el = Slugify(pretranslate=GREEK)

Examples

from slugify import Slugify, UniqueSlugify, slugify, slugify_unicode
from slugify import slugify_url, slugify_filename
from slugify import slugify_ru, slugify_de

slugify('one kožušček')                       # one-kozuscek
slugify('one two three', separator='.')       # one.two.three
slugify('one two three four', max_length=12)  # one-two-four   (12 chars)
slugify('one TWO', to_lower=True)             # one-two
slugify('one TWO', capitalize=True)           # One-TWO

slugify_filename(u'Дrаft №2.txt')             # Draft_2.txt
slugify_url(u'Дrаft №2.txt')                  # draft-2-txt

my_slugify = Slugify()
my_slugify.separator = '.'
my_slugify.pretranslate = {'я': 'i', '♥': 'love'}
my_slugify('Я ♥ борщ')                        # I.love.borshch  (custom translate)

slugify('Я ♥ борщ')                           # Ia-borshch  (standard translation)
slugify_ru('Я ♥ борщ')                        # Ya-borsch   (alternative russian translation)
slugify_unicode('Я ♥ борщ')                   # Я-борщ      (sanitize only)

slugify_de('ÜBER Über slugify')               # UEBER-Ueber-slugify

slugify_unique = UniqueSlugify(separator='_')
slugify_unique('one TWO')                     # One_TWO
slugify_unique('one TWO')                     # One_TWO_1

slugify_unique = UniqueSlugify(uids=['cellar-door'])
slugify_unique('cellar door')                 # cellar-door-1

Custom Unique Slugify Checker

from slugify import UniqueSlugify

def my_unique_check(text, uids):
    if text in uids:
        return False
    return not SomeDBClass.objects.filter(slug_field=text).exists()

custom_slugify_unique = UniqueSlugify(unique_check=my_unique_check)

# Checks the database for a matching document
custom_slugify_unique('te occidere possunt')

Running UnitTests

$ virtualenv venv
$ venv/bin/pip install -r requirements.txt
$ venv/bin/nosetests slugify
Comments
  • Update minimum unidecode version

    Update minimum unidecode version

    Unidecode has moved to semantic versioning. I don't believe there are any major changes that awesome-slugify requires.

    Unidecode release change:

    2018-01-05	unidecode 1.0.22
    	* Move to semantic version numbering, no longer following version
    	  numbers from the original Perl module. This fixes an issue with
    	  setuptools (>= 8) and others expecting major.minor.patch format.
    	  (https://github.com/avian2/unidecode/issues/13)
    	* Add transliterations for currency signs U+20B0 through U+20BF
    	  (thanks to Mike Swanson)
    	* Surround transliterations of vulgar fractions with spaces to avoid
    	  incorrect combinations with adjacent numerals
    	  (thanks to Jeffrey Gerard)
    
    opened by jwbixby 5
  • Fix for import on python 2.7.7 (windows)

    Fix for import on python 2.7.7 (windows)

    Error with awesome-slugify 1.6.4:

    python -c "from slugify import UniqueSlugify" Traceback (most recent call last): File "", line 1, in File "myvirtualenv\lib\site-packages\slugify__init__.py", line 2, in from slugify.main import Slugify, UniqueSlugify ImportError: No module named main

    opened by srault95 2
  • UniqueSlugify Improvements

    UniqueSlugify Improvements

    Hello, I'm using "awesome-slugify" for my work, and there are a few minor improvements I felt I could contribute which we were looking for on my team.

    Thanks! Greg


    1. UniqueSlugify should use sets for self.uids instead of a list, for performance reasons.

    Running the following code on the base branch (with self.uids as a list) yields:

    >>> import time
    >>> import uuid
    >>> from slugify import UniqueSlugify
    >>>
    >>> def test_time():
    ...     start = time.time()
    ...     slugify = UniqueSlugify()
    ...     for i in xrange(100000):
    ...         _ = slugify(str(uuid.uuid4()))
    ...     return time.time() - start
    ...
    >>> test_time()
    212.86210703849792
    

    Running the same code with sets:

    >>> test_time()
    10.824954986572266
    
    1. Often, uids are stored in a database or external key/value system. It's helpful to have an option to override the uniqueness check without having to load all the uids into memory. For example, supposing I have a Django project with a slug field, instead of having to do:
    from django_blog.models import BlogPost
    from slugify import UniqueSlugify
    
    slugify = UniqueSlugify(uids=BlogPost.objects.values_list('url_slug', flat=True))
    

    I can check with a lightweight existence call to the DB by overriding the check:

    from slugify import UniqueSlugify
    
    def my_unique_check(text, uids):
        if text in uids:
            return False
        return not BlogPost.objects.filter(url_slug=text).exists()
    
    custom_slugify_unique = UniqueSlugify(unique_check=my_unique_check)
    custom_slugify_unique('te occidere possunt')
    
    1. Also added some documentation on running the unit tests.
    opened by gthole 2
  • not working in python 3

    not working in python 3

    Please make this library compatible with Python 3. Right now it doesn't work because of at least several syntax errors in some string literals, and an iteration over a changing dict issue.

    opened by irmen 2
  • Please remove capitalizing of first character in get_pretranslate function

    Please remove capitalizing of first character in get_pretranslate function

    Great project! Thank you so much!

    However, when using the pretranslations, it forces capitalization. Is there any reason for this (something unicode related) or can I submit a pull request to maintain the case of the entire word?

    opened by pydanny 2
  • Fix the import path for get_slugify

    Fix the import path for get_slugify

    At the moment, I can't do:

    from slugify import get_slugify
    

    I get an import error because of how slugify.__init__.py is defined. Instead I have to do the following:

    from slugify.main import get_slugify
    

    I would like to either submit a patch to slugify.__init__.py or correct the documentation. @dimka665, what approach would you prefer?

    opened by pydanny 2
  • Add test cases with numbers in them.

    Add test cases with numbers in them.

    We use awesome-slugify for Wok and it's mostly great, but I've found that it errors on numeric slugs (in our case, 404).

    These test cases demonstrate what I'd naively assume to be the correct slugification behavior for alphanumerics, but I'm PR-ing to verify that before spending a lot of time modifying Slugify's behavior to meet these expectations in case I'm wrong.

    opened by edunham 1
  • Add simple implementation with Travis CI

    Add simple implementation with Travis CI

    Check out passing tests of my fork on Travis https://travis-ci.org/jpadilla/awesome-slugify

    If you accept this, you just have to signup to Travis and setup your accounts https://travis-ci.org/profile. Then add a badge to the README.md.

    opened by jpadilla 1
  • Fix to_lower with unicode text

    Fix to_lower with unicode text

    I noticed that when trying to do something like:

    slugify('自転車', to_lower=True)
    

    I got Zi-Zhuan-Che instead of the expected zi-zhuan-che. Effectively I introduced a test which failed, implemented a fix, and all tests kept passing.

    opened by jpadilla 1
  • allow Slugify to take callable pretranslate

    allow Slugify to take callable pretranslate

    Slugify::set_pretranslate should accept a dictionary, a callable, or None, but as currently implemented, it only accepts a dict or None. This PR allows set_translate to take a callable as well. All tests passing.

    opened by jmcarp 1
  • Exception StopIteration on empty strings '' with max_length or separator

    Exception StopIteration on empty strings '' with max_length or separator

    >>> from slugify import slugify
    >>> slugify('', max_length=40, separator='_')
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python2.7/dist-packages/slugify/main.py", line 108, in slugify
        text = join_words(words, separator, max_length)
      File "/usr/lib/python2.7/dist-packages/slugify/main.py", line 85, in join_words
        text = next(words)    # text = words.pop(0)
    StopIteration
    
    opened by sebest 1
  • Incorrect Japanese transliteration for っ

    Incorrect Japanese transliteration for っ

    っ is U+3063 HIRAGANA LETTER SMALL TU, which is different than つ U+3064 HIRAGANA LETTER TU in that the phonetic transliteration of it is a glottal stop; the English equivalent is doubling the consonant-sound of the next mora.

    For example, ほっこり should be transliterated as 'hokkori', but awesome-slugify incorrectly renders it 'hotsukori' (as if the っ were a つ):

    >>> slugify('ほっこり')
    'hotsukori'
    

    See also https://translate.google.com/?sl=ja&tl=en&text=%E3%81%BB%E3%81%A3%E3%81%93%E3%82%8A&op=translate for the use of this character (and https://translate.google.com/?sl=ja&tl=en&text=%E3%81%BB%E3%81%A4%E3%81%93%E3%82%8A%0A%0A&op=translate to see what the large tsu does instead).

    opened by fluffy-critter 0
  • Clash with zacharyvoase/slugify

    Clash with zacharyvoase/slugify

    The project at https://github.com/zacharyvoase/slugify named "slugify" also has the module name "slugify".

    pip install slugify

    import slugify slugify.slugify(u"Héllø Wörld")

    If you add both packages to your requirements.txt, what happens when you import "slugify"?

    opened by Chris2048 4
  • Fix DeprecationWarnings in newer Pythons (3.6+)

    Fix DeprecationWarnings in newer Pythons (3.6+)

    I see these warnings each time I run:

    ...python3.6/site-packages/slugify/main.py:65
      ...python3.6/site-packages/slugify/main.py:65: DeprecationWarning: invalid escape sequence \p
        '''
    
    ...python3.6/site-packages/slugify/main.py:98
      ...python3.6/site-packages/slugify/main.py:98: DeprecationWarning: invalid escape sequence \L
        PRETRANSLATE = re.compile(u'(\L<options>)', options=convert_dict)
    
    ...python3.6/site-packages/slugify/main.py:140
      ...python3.6/site-packages/slugify/main.py:140: DeprecationWarning: invalid escape sequence \p
        unwanted_chars_re = u'[^\p{{AlNum}}{safe_chars}]+'.format(safe_chars=re.escape(self._safe_chars or ''))
    
    ...python3.6/site-packages/slugify/main.py:144
      ...python3.6/site-packages/slugify/main.py:144: DeprecationWarning: invalid escape sequence \p
        unwanted_chars_and_words_re = unwanted_chars_re + u'|(?<!\p{AlNum})(?:\L<stop_words>)(?!\p{AlNum})'
    

    Perhaps this is the problem: https://stackoverflow.com/questions/50504500/deprecationwarning-invalid-escape-sequence-what-to-use-instead-of-d

    opened by mcarans 0
  • Update requirements.txt

    Update requirements.txt

    regex==2018.11.6 breaks this package:

    $ pip install regex==2018.11.6
    [...]
    $ python
    Python 2.7.10 (default, Oct  6 2017, 22:29:07)
    [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.31)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import slugify
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/slugify/__init__.py", line 2, in <module>
        from slugify.main import Slugify, UniqueSlugify
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/slugify/main.py", line 68, in <module>
        class Slugify(object):
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/slugify/main.py", line 70, in Slugify
        upper_to_upper_letters_re = re.compile(UPPER_TO_UPPER_LETTERS_RE, re.VERBOSE | re.VERSION1)
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/regex.py", line 345, in compile
        return _compile(pattern, flags, kwargs)
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/regex.py", line 486, in _compile
        parsed = _parse_pattern(source, info)
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 388, in _parse_pattern
        branches = [parse_sequence(source, info)]
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 413, in parse_sequence
        element = parse_paren(source, info)
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 823, in parse_paren
        subpattern = _parse_pattern(source, info)
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 388, in _parse_pattern
        branches = [parse_sequence(source, info)]
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 410, in parse_sequence
        sequence.append(parse_escape(source, info, False))
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 1186, in parse_escape
        return parse_property(source, info, ch == "p", in_set)
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 1341, in parse_property
        prop = lookup_property(prop_name, name, positive != negate, source)
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 1603, in lookup_property
        value = standardise_name(value)
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 1593, in standardise_name
        return ascii_upper("".join(ch for ch in name if ch not in "_- "))
      File "/Users/artur/venv/qq/lib/python2.7/site-packages/_regex_core.py", line 1586, in ascii_upper
        return s.translate(upper_trans)
    TypeError: character mapping must return integer, None or unicode
    
    opened by arturh 0
  • UniqueSlugify will exceed max_length if it adds digits to make slug unique

    UniqueSlugify will exceed max_length if it adds digits to make slug unique

    Python 3.6.4 (default, Mar  9 2018, 23:15:03)
    [GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from slugify import UniqueSlugify
    >>> s = UniqueSlugify(to_lower=True, max_length=3)
    >>> s("Hello World")
    'hel'
    >>> s("Hello World")
    'hel-1'
    

    I would expect something like hel, h-1, etc.

    opened by iandees 0
Releases(v1.4)
Problem 5: Fermat near-misses

Problem 5: Fermat near-misses fermatnearmiss This is a script that computes fermat nearm misses when the -f option is set and requires users to input

CHRIS BYRON (Int0x80) 1 Jan 08, 2022
Python bilgilerimi eğlenceli bir şekilde hatırlamak ve daha da geliştirmek için The Big Book of Small Python Projects isimli bir kitap almıştım.

Python bilgilerimi eğlenceli bir şekilde hatırlamak ve daha da geliştirmek için The Big Book of Small Python Projects isimli bir kitap almıştım. Bu repo kitaptaki örnek programları çalıştığım oyun al

Burak Selim Senyurt 22 Oct 26, 2022
The code submitted for the Analytics Vidhya Jobathon - February 2022

Introduction On February 11th, 2022, Analytics Vidhya conducted a 3-day hackathon in data science. The top candidates had the chance to be selected by

11 Nov 21, 2022
Using Python to parse through email logs received through several backup systems.

outlook-automated-backup-control Backup monitoring on a mailbox: In this mailbox there will be backup logs. The identification will based on the follo

Connor 2 Sep 28, 2022
A very basic ciphering/deciphering tool

ckrett-python-library This is an useful python library for people who care about privacy, this library is useful to cipher and decipher text using 4 s

SasiVatsal 8 Oct 18, 2022
Python Common things by Problem Fighter Library, (Exception, Debug Log, etc.)

In the name of God, the Most Gracious, the Most Merciful. PF-PY-Common Documentation Install and update using pip: pip install -U xxxx Please find the

Problem Fighter 3 Jan 15, 2022
With Christmas and New Year ahead, it is time for some festive coding. Here is a Christmas Card for you all!

Christmas Card With Christmas and New Year ahead, it is time for some festive coding! Here is a Christmas Card for you all! NOTE: I have not made this

CodeMaster7000 1 Dec 25, 2021
An easy FASTA object handler, reader, writer and translator for small to medium size projects without dependencies.

miniFASTA An easy FASTA object handler, reader, writer and translator for small to medium size projects without dependencies. Installation Using pip /

Jules Kreuer 3 Jun 30, 2022
It was created to conveniently respond to events such as donation, follow, and hosting using the Alert Box provided by twip to streamers

This library is not an official library of twip. It was created to conveniently respond to events such as donation, follow, and hosting using the Alert Box provided by twip to streamers.

junah201 8 Nov 19, 2022
Palestra sobre desenvolvimento seguro de imagens e containers para a DockerCon 2021 sala Brasil

Segurança de imagens e containers direto na pipeline Palestra sobre desenvolvimento seguro de imagens e containers para a DockerCon 2021 sala Brasil.

Fernando Guisso 10 May 19, 2022
Comics/doujinshi reader application. Web-based, will work on desktop and tablet devices with swipe interface.

Yomiko Comics/doujinshi reader application. Web-based, will work on desktop and tablet devices with swipe interface. Scans one or more directories of

Kyubi Systems 26 Aug 10, 2022
A Python library that helps data scientists to infer causation rather than observing correlation.

A Python library that helps data scientists to infer causation rather than observing correlation.

QuantumBlack Labs 1.7k Jan 04, 2023
Lock a program and kills it indefinitely if it is started.

Kill By Lock Lock a program and kills it indefinitely if it is started. How start it? It' simple, you just have to double-click on the python file (.p

1 Jan 12, 2022
A curses based mpd client with basic functionality and album art.

Miniplayer A curses based mpd client with basic functionality and album art. After installation, the player can be opened from the terminal with minip

Tristan Ferrua 102 Dec 24, 2022
This app converts an pdf file into the audio file.

PDF-to-Audio This app takes an pdf as an input and convert it into audio, and the library text-to-speech starts speaking the preffered page given in t

Ojas Barawal 3 Aug 04, 2021
Stop ask your soraka to ult you, just ult yourself

Lollo's auto-ultimate script Are you tired of your low elo friend who can't ult you with soraka when you ask for it? Use Useless Support and just ult

9 Oct 20, 2022
Task dispatcher for Postgres

Features a task being ran as an OS process supports task queue with priority and process limit per node fully database driven (a worker and task can b

2 Dec 06, 2021
Parser for RISC OS Font control characters in Python

RISC OS Font control parsing in Python This repository contains a class (FontControlParser) for parsing font control codes from a byte squence, in Pyt

Charles Ferguson 1 Nov 02, 2021
Python for Microscopists and other image processing enthusiasts

The YouTube channel associated with this code walks you through the entire process of learning to code in Python; all the way from basics to advanced machine learning and deep learning.

Dr. Sreenivas Bhattiprolu 2.3k Jan 01, 2023
🟥This is an overview of how to set up and use DataStore3 in your Roblox experiences

Welcome to DataStore3 👋 This is an overview of how to set up and use DataStore3 in your Roblox experiences What is it? 🤔 DataStore3 is a service tha

Reece Harris 7 Aug 19, 2022