A generator library for concise, unambiguous and URL-safe UUIDs.

Overview

Description

shortuuid is a simple python library that generates concise, unambiguous, URL-safe UUIDs.

Often, one needs to use non-sequential IDs in places where users will see them, but the IDs must be as concise and easy to use as possible. shortuuid solves this problem by generating uuids using Python's built-in uuid module and then translating them to base57 using lowercase and uppercase letters and digits, and removing similar-looking characters such as l, 1, I, O and 0.

https://travis-ci.org/skorokithakis/shortuuid.svg?branch=master

Installation

To install shortuuid you need:

  • Python 2.5 or later in the 2.x line (earlier than 2.6 not tested), or any 3.x.

If you have the dependencies, you have multiple options of installation:

Usage

To use shortuuid, just import it in your project like so:

>>> import shortuuid

You can then generate a short UUID:

>>> shortuuid.uuid()
'vytxeTZskVKR7C7WgdSP3d'

If you prefer a version 5 UUID, you can pass a name (DNS or URL) to the call and it will be used as a namespace (uuid.NAMESPACE_DNS or uuid.NAMESPACE_URL) for the resulting UUID:

>>> shortuuid.uuid(name="example.com")
'wpsWLdLt9nscn2jbTD3uxe'
>>> shortuuid.uuid(name="http://example.com")
'c8sh5y9hdSMS6zVnrvf53T'

You can also generate a cryptographically secure random string (using os.urandom(), internally) with:

>>> shortuuid.ShortUUID().random(length=22)
'RaF56o2r58hTKT7AYS9doj'

To see the alphabet that is being used to generate new UUIDs:

>>> shortuuid.get_alphabet()
'23456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz'

If you want to use your own alphabet to generate UUIDs, use set_alphabet():

>>> shortuuid.set_alphabet("aaaaabcdefgh1230123")
>>> shortuuid.uuid()
'0agee20aa1hehebcagddhedddc0d2chhab3b'

shortuuid will automatically sort and remove duplicates from your alphabet to ensure consistency:

>>> shortuuid.get_alphabet()
'0123abcdefgh'

If the default 22 digits are too long for you, you can get shorter IDs by just truncating the string to the desired length. The IDs won't be universally unique any longer, but the probability of a collision will still be very low.

To serialize existing UUIDs, use encode() and decode():

>>> import uuid ; u = uuid.uuid4() ; u
UUID('6ca4f0f8-2508-4bac-b8f1-5d1e3da2247a')
>>> s = shortuuid.encode(u) ; s
'cu8Eo9RyrUsV4MXEiDZpLM'
>>> shortuuid.decode(s) == u
True
>>> short = s[:7] ; short
'cu8Eo9R'
>>> h = shortuuid.decode(short)
UUID('00000000-0000-0000-0000-00b8c0b9f952')
>>> shortuuid.decode(shortuuid.encode(h)) == h
True

Class-based usage

If you need to have various alphabets per-thread, you can use the ShortUUID class, like so:

>>> su = shortuuid.ShortUUID(alphabet="01345678")
>>> su.uuid()
'034636353306816784480643806546503818874456'
>>> su.get_alphabet()
'01345678'
>>> su.set_alphabet("21345687654123456")
>>> su.get_alphabet()
'12345678'

Command-line usage

shortuuid provides a simple way to generate a short UUID in a terminal:

$ python3 -m shortuuid
fZpeF6gcskHbSpTgpQCkcJ

(Replace python3 with py if you are using Windows)

Compatibility note

Versions of ShortUUID prior to 1.0.0 generated UUIDs with their MSB last, i.e. reversed. This was later fixed, but if you have some UUIDs stored as a string with the old method, you need to pass legacy=True to decode() when converting your strings back to UUIDs.

That option will go away in the future, so you will want to convert your UUIDs to strings using the new method. This can be done like so:

>>> new_uuid_str = encode(decode(old_uuid_str, legacy=True))

License

shortuuid is distributed under the BSD license.

Comments
  • LSB-first encoding makes lexicographic ordering hard

    LSB-first encoding makes lexicographic ordering hard

    Let's say I want to group a number of shortuuids into several non-overlapping partitions and use a string-based comparison that defines each partition, e.g. "aaa" <= shortuuid < "bbb". Since shortuuids are LSB-first, this isn't possible, since 0 corresponds to '2222222222222222222222' and 1 corresponds to '3222222222222222222222'. With UUIDs generated with str(uuid.uuid4()) this would be '00000000-0000-0000-0000-000000000000' for 0 and '00000000-0000-0000-0000-000000000001' for 1, which is useable for lexicographic ordering based on their integer representation. Is there a specific reason why shortuuids are LSB-first? In my case it complicates things, but are there advantages I'm ignoring?

    opened by letmaik 22
  • ShortUUID should not sort the alphabet

    ShortUUID should not sort the alphabet

    I am attempting to convert a shortuuid generated using node.js and the Flicker base 58 alphabet:

    shortid = 'apECAPA6RdHB6HB51FwsdN'
    FLICKR_BASE58 = '123456789abcdefghijkmnopqrstuvwxyzABCDEFGHJKLMNPQRSTUVWXYZ'
    uuid = shortuuid.ShortUUID(alphabet=FLICKR_BASE58).decode(shortid)
    
    

    Unfortunately this doesn't work:

    Traceback (most recent call last):
      File "<ipython-input-54-88d3ac3dd885>", line 3, in <module>
        uuid = shortuuid.ShortUUID(alphabet=FLICKR_BASE58).decode(shortid)
      File "/Users/scotts/.pyenv/versions/python38/lib/python3.8/site-packages/shortuuid/main.py", line 79, in decode
        return _uu.UUID(int=string_to_int(string, self._alphabet))
      File "/Users/scotts/.pyenv/versions/3.8.7/lib/python3.8/uuid.py", line 205, in __init__
        raise ValueError('int is out of range (need a 128-bit value)')
    ValueError: int is out of range (need a 128-bit value)
    

    It turns out that ShortUUID is sorting the alphabet before using it. This is incorrect and leads to this failure. It is possible to correctly decode the id like this:

    import uuid as _uu
    uuid = _uu.UUID(int=string_to_int(shortid, FLICKR_BASE58))
    

    This is a backwards compatibility issue, so switching to an unsorted implementation will probably require a compatibility flag similar to legacy.

    opened by snstanton 17
  • Generating invalid ShortUUIDs

    Generating invalid ShortUUIDs

    I'm calling shortuuid.uuid() in our unit test which run regularly and I'm seeing it generate some invalid UUIDs

    Such as 5vHqLc9etTWW3AVrEJvBez.

    The library seems to think this is a valid UUID

    >>> shortuuid.decode("5vHqLc9etTWW3AVrEJvBez")
    UUID('160427e7-f28e-40ee-9f7a-d473721488c8')
    

    But your website does not: https://shortuuid.com/

    I'm also validating this with some Elixir libraries and they are reporting that it's invalid as well.

    Any help would be greatly appreciated.

    opened by kevinkirkup 14
  • Added PaddedShortUUID for constant length short uuid strings

    Added PaddedShortUUID for constant length short uuid strings

    Small enough UUID integers can create shorter uuid strings (21 character strings are generated ~1-2% of the time with the default alphabet). When you want a fixed length ID generated, use this PaddedShortUUID class instead.

    opened by kevinastone 13
  • Add MSB-first to https://shortuuid.com/

    Add MSB-first to https://shortuuid.com/

    I'm not sure how is the owner of https://shortuuid.com/, but I think this is the right place to post the request.

    I'm heavly using shortuuid in the system, so I'm using https://shortuuid.com/ to convert UUID<->shortuuid. However, I'm using shortuuid v1.0.9 which has MSB-first decoding while the site use legacy decoding, so I'm using string reverse tool to get the MSB shortuuid.

    It would be great if you can add a new field for MSB or a radio button to switch to MSB decoding to https://shortuuid.com/

    Thanks!

    opened by MotasemAghbar 12
  • shortuuid performance and sorting

    shortuuid performance and sorting

    I was playing with our Java UUID shortner I wrote several years ago and I decided to see what other folks are doing and came across library.

    Anyway I have few critiques.

    The algorithm as I understand turns the UUID into one giant 128 bit integer and divmods appending the remainder to a string. The most significant part of the UUID is appended first but the contents of the most significant has the most significant right most. For example base10 a "100" would be 001 or in base57 m3.

    There are two problems to this approach:

    • Sorting is no longer the same. That is if you have a list of UUIDs and list of corresponding shortuuids the sort order will not be same. This is because it follows the bit order which makes since for a generic encoding algorithm but I don't think its good for a UUID shortner.
    • The algorithm is very slow on Java. Or at least this implementation which appears to be a clone of the golang https://github.com/hsingh/java-shortuuid and mostly follows the python version.

    An easy replacement for the Java one is to use Bitcoin's Base58 and just change the alphabet which is originally what I used to use for UUID shortening. However Base58 divmods byte by byte and does not construct a full 128 bit integer.

    Still Base58 is actually still not that fast compared to say Base64 and that is obviously because of power of 2 division.

    My algorithm is almost as fast as base 64. My algorithm works by divmoding longs (64 bit numbers in java) one at a time. This is fine because a UUID is two 64 bit numbers. Secondly there are tricks divmoding 64 bit numbers super fast even if they are not powers of 2 divisors. Secondly it appends the most significant left to write to preserve lexicographical sort order. Padding is required but based on my tests only like ~2% of UUIDs have less than 22 characters (I was too lazy to do the math).

    Anyway before I go set an opensource repository and try to port it to python I wanted to get some feedback. Is this a bad idea?

    opened by agentgt 12
  • Using pkg_resources makes package load really slow

    Using pkg_resources makes package load really slow

    Heya, this change really hurts package loading times 😬 https://github.com/skorokithakis/shortuuid/blob/bcd01ceaf2419d874b8543e6d7c28e2c56da7a91/shortuuid/init.py#L13

    For example, with our CLI this change has meant an increase in load time from ~0.3 seconds to ~0.8 seconds!

    pkg_resources is essentially deprecated, in part, for this very reason; replaced by https://docs.python.org/3/library/importlib.metadata.html and the https://github.com/python/importlib_metadata backport

    opened by chrisjsewell 8
  • Duplicated shord uuid

    Duplicated shord uuid

    I'm using shortuuid.ShortUUID().random(length=22) for generating short-uuids in a django application on Heroku.

    In already 2 cases I found a duplicated uuid. I haven't been able to reproduce whenever I want the error yet.

    opened by leomrocha 8
  • add py.typed to MANIFEST.in, re-export API

    add py.typed to MANIFEST.in, re-export API

    Thanks for shortuuid!

    This PR:

    • [x] updates MANIFEST.in so that the py.typed file is included in the sdist on PyPI, useful for downstream re-builders.
    • [x] populates __all__ in __init__.py to avoid some mypy warnings
      • [x] allows removing the noqa comments
    opened by bollwyvl 7
  • Unclear how to use UUID5, namespace

    Unclear how to use UUID5, namespace

    My understanding of UUID v5 is you can provide a namespace and a value to get a deterministic hash for that namespace + value.

    I see you can provide shortuuid.uuid a name argument, and I see in the code how that maps to uuid5 in the code... but it's unclear to me what to do from there.

    I see that providing the same name argument produces the same output (as expected!). How do you now combine a namespace and value to get a deterministic output, such that func(namespace, value) produces the same output every time (where namespace remains the same and value changes)?

    Perhaps I'm misunderstanding something. Thanks.

    opened by connerxyz 7
  • New release has broken existing URLs

    New release has broken existing URLs

    I store UUIDs in my database and render to short UUIDs in the UI, especially in URLs. Last week's new releases have broken all my existing URLs because of the MSB issue.

    While the legacy=True flag is available, the promise that it will go away in future is not comforting. There is absolutely nothing I can do to withdraw the thousands of existing URLs I have in circulation (as will be the case for anyone who has used ShortUUID in production).

    I'd like the option to continue generating legacy short UUIDs forever. The MSB order is irrelevant anyway in a rendered string.

    opened by jace 7
Releases(v1.0.0)
  • v1.0.0(Mar 6, 2020)

    NOTE: THIS IS A BREAKING RELEASE. See the compatibility note in the README before upgrading.

    v1.0.0 (2020-03-06)

    Features

    • Drop support for Python before 3.5. [Stavros Korokithakis]

    • Add simple command-line interface (#43) [Éric Araujo]

    Fixes

    • Make encode and decode MSB-first (#36) [Keane Nguyen]

    • Make the URL check more robust (fixes #32) [Stavros Korokithakis]

    v0.5.0 (2017-02-19)

    Features

    • Make int_to_string and string_to_int available globally. [Stavros Korokithakis]
    Source code(tar.gz)
    Source code(zip)
  • v0.5.0(Feb 20, 2017)

  • v0.4.3(Jan 13, 2016)

Owner
Stavros Korokithakis
I love writing code, making stupid devices and writing about writing code and making stupid devices.
Stavros Korokithakis
Extract price amount and currency symbol from a raw text string

price-parser is a small library for extracting price and currency from raw text strings.

Scrapinghub 252 Dec 31, 2022
Utility for Text Normalisation or Inverse Normalisation

Text Processor Text Normalisation or Inverse Normalisation for Indonesian, e.g. measurements "123 kg" - "seratus dua puluh tiga kilogram" Currency/Mo

Cahya Wirawan 2 Aug 11, 2022
A simple Python module for parsing human names into their individual components

Name Parser A simple Python (3.2+ & 2.6+) module for parsing human names into their individual components. hn.title hn.first hn.middle hn.last hn.suff

Derek Gulbranson 574 Dec 20, 2022
Python character encoding detector

Chardet: The Universal Character Encoding Detector Detects ASCII, UTF-8, UTF-16 (2 variants), UTF-32 (4 variants) Big5, GB2312, EUC-TW, HZ-GB-2312, IS

Character Encoding Detector 1.8k Jan 08, 2023
Hamming code generation, error detection & correction.

Hamming code generation, error detection & correction.

Farhan Bin Amin 2 Jun 30, 2022
Phone Number formatting for PlaySMS Platform - BulkSMS Platform

BulkSMS-Number-Formatting Phone Number formatting for PlaySMS Platform - BulkSMS Platform. Phone Number Formatting for PlaySMS Phonebook Service This

Edwin Senunyeme 1 Nov 08, 2021
A Python app which can convert normal text to Handwritten text.

Text to HandWritten Text ✍️ Converter Watch Tutorial for this project Usage:- Clone my repository. Open CMD in working directory. Run following comman

Kushal Bhavsar 5 Dec 11, 2022
A production-ready pipeline for text mining and subject indexing

A production-ready pipeline for text mining and subject indexing

UF Open Source Club 12 Nov 06, 2022
A Python package to facilitate research on building and evaluating automated scoring models.

Rater Scoring Modeling Tool Introduction Automated scoring of written and spoken test responses is a growing field in educational natural language pro

ETS 59 Oct 10, 2022
Umamusume story patcher with python

umamusume-story-patcher How to use Go to your umamusume folder, usually C:\Users\user\AppData\LocalLow\Cygames\umamusume Make a mods folder and clon

8 May 07, 2022
Python Q&A for Network Engineers

Q & A I am often asked questions about how to solve this or that problem, and I decided to post these questions and solutions here, in case it is also

Natasha Samoylenko 30 Nov 15, 2022
The project is investigating methods to extract human-marked data from document forms such as surveys and tests.

The project is investigating methods to extract human-marked data from document forms such as surveys and tests. They can read questions, multiple-choice exam papers, and grade.

Harry 5 Mar 27, 2022
WorldCloud Orçamento de Estado 2022

World Cloud Orçamento de Estado 2022 What it does This script creates a worldcloud, masked on a image, from a txt file How to run it? Install all libr

Jorge Gomes 2 Oct 12, 2021
strbind - lapidary text converter for translate an text file to the C-style string

strbind strbind - lapidary text converter for translate an text file to the C-style string. My motivation is fast adding large text chunks to the C co

Mihail Zaytsev 1 Oct 22, 2021
A simple text editor for linux

wolf-editor A simple text editor for linux Installing using Deb Package Download newest package from releases CD into folder where the downloaded acka

Focal Fossa 5 Nov 30, 2021
This is a text summarizing tool written in Python

Summarize Written by: Ling Li Ya This is a text summarizing tool written in Python. User Guide Some things to note: The application is accessible here

Marcus Lee 2 Feb 18, 2022
The bot creates hashtags for user's texts in Russian and English.

telegram_bot_hashtags The bot creates hashtags for user's texts in Russian and English. It is a simple bot for creating hashtags. NOTE file config.py

Yana Davydovich 2 Feb 12, 2022
Converts a Bangla numeric string to literal words.

Bangla Number in Words Converts a Bangla numeric string to literal words. Install $ pip install banglanum2words Usage

Syed Mostofa Monsur 3 Aug 29, 2022
一个可以可以统计群组用户发言,并且能将聊天内容生成词云的机器人

当前版本 v2.2 更新维护日志 更新维护日志 有问题请加群组反馈 Telegram 交流反馈群组 点击加入 演示 配置要求 内存:1G以上 安装方法 使用 Docker 安装 Docker官方安装

机器人总动员 117 Dec 29, 2022
Implementation of hashids (http://hashids.org) in Python. Compatible with Python 2 and Python 3

hashids for Python 2.7 & 3 A python port of the JavaScript hashids implementation. It generates YouTube-like hashes from one or many numbers. Use hash

David Aurelio 1.4k Jan 02, 2023