A pickled object field for Django

Overview

django-picklefield

License Latest Version Build Status Coverage Status Supported Python Versions Wheel Status

About

django-picklefield provides an implementation of a pickled object field. Such fields can contain any picklable objects.

The implementation is taken and adopted from Django snippet #1694 by Taavi Taijala, which is in turn based on Django snippet #513 by Oliver Beattie.

django-picklefield is available under the MIT license.

Usage

First of all, you need to have django-picklefield installed; for your convenience, recent versions should be available from PyPI.

To use, just define a field in your model:

>>> from picklefield.fields import PickledObjectField
... class SomeObject(models.Model):
...     args = PickledObjectField()

and assign whatever you like (as long as it's picklable) to the field:

>>> obj = SomeObject()
>>> obj.args = ['fancy', {'objects': 'inside'}]
>>> obj.save()

Notes

If you need to serialize an object with a PickledObjectField for transmission to the browser, you may need to subclass the field and override the value_to_string() method. Currently pickle fields are serialized as base64-encoded pickles, which allows reliable deserialization, but such a format is not convenient for parsing in the browser. By overriding value_to_string() you can choose a more convenient serialization format.

Fields now accept the boolean key word argument copy, which defaults to True. The copy is necessary for lookups to work correctly. If you don't care about performing lookups on the picklefield, you can set copy=False to save on some memory usage. This an be especially beneficial for very large object trees.

Running tests

The full test suite can be run with Tox:

>>> pip install tox
>>> tox

Original notes

Here are the notes by taavi223, the original author:

Incredibly useful for storing just about anything in the database (provided it is Pickle-able, of course) when there isn't a 'proper' field for the job.

PickledObjectField is database-agnostic, and should work with any database backend you can throw at it. You can pass in any Python object and it will automagically be converted behind the scenes. You never have to manually pickle or unpickle anything. Also works fine when querying; supports exact, in, and isnull lookups. It should be noted, however, that calling QuerySet.values() will only return the encoded data, not the original Python object.

This PickledObjectField has a few improvements over the one in snippet #513.

This one solves the DjangoUnicodeDecodeError problem when saving an object containing non-ASCII data by base64 encoding the pickled output stream. This ensures that all stored data is ASCII, eliminating the problem.

PickledObjectField will now optionally use zlib to compress (and uncompress) pickled objects on the fly. This can be set per-field using the keyword argument "compress=True". For most items this is probably not worth the small performance penalty, but for Models with larger objects, it can be a real space saver.

You can also now specify the pickle protocol per-field, using the protocol keyword argument. The default of 2 should always work, unless you are trying to access the data from outside of the Django ORM.

Worked around a rare issue when using the cPickle and performing lookups of complex data types. In short, cPickle would sometimes output different streams for the same object depending on how it was referenced. This of course could cause lookups for complex objects to fail, even when a matching object exists. See the docstrings and tests for more information.

You can now use the isnull lookup and have it function as expected. A consequence of this is that by default, PickledObjectField has null=True set (you can of course pass null=False if you want to change that). If null=False is set (the default for fields), then you wouldn't be able to store a Python None value, since None values aren't pickled or encoded (this in turn is what makes the isnull lookup possible).

You can now pass in an object as the default argument for the field without it being converted to a unicode string first. If you pass in a callable though, the field will still call it. It will not try to pickle and encode it.

You can manually import dbsafe_encode and dbsafe_decode from fields.py if you want to encode and decode objects yourself. This is mostly useful for decoding values returned from calling QuerySet.values(), which are still encoded strings.

Note: If you are trying to store other django models in the PickledObjectField, please see the comments for a discussion on the problems associated with doing that. The easy solution is to put django models into a list or tuple before assigning them to the PickledObjectField.

Update 9/2/09: Fixed the value_to_string method so that serialization should now work as expected. Also added deepcopy back into dbsafe_encode, fixing #4 above, since deepcopy had somehow managed to remove itself. This means that lookups should once again work as expected in all situations. Also made the field editable=False by default (which I swear I already did once before!) since it is never a good idea to have a PickledObjectField be user editable.

Changes

Changes in version 3.0.0

  • Allowed default pickle protocol to be overriden using the PICKLEFIELD_DEFAULT_PROTOCOL setting.
  • Dropped support for Python 2.
  • Added testing against Django 3.0.
  • Dropped support for Django 1.11.

Changes in version 2.1.0

  • Added official support for Django 2.2 (thanks to joehybird).
  • Dropped support for Django 2.0 and 2.1 (thanks to joehybird).
  • Dropped support for Python 3.4 (thanks to joehybidd).

Changes in version 2.0.0

  • Silenced RemovedInDjango30Warning warnings on Django 2.0+ (thanks to canarduck).
  • Restructured project directories.
  • Disallowed the usage of empty strings for PickledObjectField. That makes .save(), .create(), etc. raise IntegrityError if null is not True and no default value was specified like built-in fields do (thanks to Attila-Mihaly Balazs).
  • Added a check for mutable default values to PickledObjectField.

Changes in version 1.1.0

  • Added support for Django 2.1 and dropped support for Django < 1.11.

Changes in version 1.0.0

  • Added a new option to prevent a copy of the object before pickling: copy=True
  • Dropped support for Django 1.4
  • Dropped support for Django 1.7
  • Dropped support for Python 3.2
  • Added support for Python 3.6

Changes in version 0.3.2

  • Dropped support for Django 1.3.
  • Dropped support for Python 2.5.
  • Silenced deprecation warnings on Django 1.8+.

Changes in version 0.3.1

  • Favor the built in json module (thanks to Simon Charette).

Changes in version 0.3.0

  • Python 3 support (thanks to Rafal Stozek).

Changes in version 0.2.0

  • Allow pickling of subclasses of django.db.models.Model (thanks to Simon Charette).

Changes in version 0.1.9

  • Added connection and prepared parameters to get_db_prep_value() too (thanks to Matthew Schinckel).

Changes in version 0.1.8

  • Updated link to code repository.

Changes in version 0.1.7

  • Added connection and prepared parameters to get_db_prep_lookup() to get rid of deprecation warnings in Django 1.2.

Changes in version 0.1.6

Changes in version 0.1.5

  • Added support for South.
  • Changed default to null=False, as is common throughout Django.

Changes in version 0.1.4

  • Updated copyright statements.

Changes in version 0.1.3

  • Updated serialization tests (thanks to Michael Fladischer).

Changes in version 0.1.2

  • Added Simplified BSD licence.

Changes in version 0.1.1

  • Added test for serialization.
  • Added note about JSON serialization for browser.
  • Added support for different pickle protocol versions (thanks to Michael Fladischer).

Changes in version 0.1

  • First public release.

Feedback

There is a home page <http://github.com/gintas/django-picklefield> with instructions on how to access the code repository.

Send feedback and suggestions to [email protected] .

Owner
Gintautas Miliauskas
CTO / co-founder at Mavenoid
Gintautas Miliauskas
Automated image processing for Django. Currently v4.0

ImageKit is a Django app for processing images. Need a thumbnail? A black-and-white version of a user-uploaded image? ImageKit will make them for you.

Matthew Dapena-Tretter 2.1k Dec 17, 2022
Chatbot for ordering and tracking a Pizza.

Pizza Chatbot To start the app, follow the below steps: Clone the repo using the below command: git clone Shreya Shah 1 Jul 15, 2021

A Django backed for PostgreSQL using Psycopg 3

A Django backend for PostgreSQL using Psycopg 2 The backend passes the entire Django test suite, but it needs a few modifications to Django and to i

Daniele Varrazzo 42 Dec 16, 2022
AUES Student Management System Developed for laboratory works №9 Purpose using Python (Django).

AUES Student Management System (L M S ) AUES Student Management System Developed for laboratory works №9 Purpose using Python (Django). I've created t

ANAS NABIL 2 Dec 06, 2021
A modern looking portfolio build with Django.

Django Portfolio A portfolio template using html/css/js in the frontend and Django as the backend framework. Cool features: smooth scrolling responsiv

1 Jan 19, 2022
A pickled object field for Django

django-picklefield About django-picklefield provides an implementation of a pickled object field. Such fields can contain any picklable objects. The i

Gintautas Miliauskas 167 Oct 18, 2022
Wagtail - Vue - Django : The initial environment of full-stack local dev web app with wagtail and vue

Wagtail - Vue - Django : The initial environment of full-stack local dev web app with wagtail and vue. A demo to show how to use .vue files inside django app.

Quang PHAM 2 Oct 20, 2022
simple project management tool for educational purposes

Taskcamp This software is used for educational and demonstrative purposes. It's a simple project management tool powered by Django Framework Install B

Ilia Dmitriev 6 Nov 08, 2022
A Django app to accept payments from various payment processors via Pluggable backends.

Django-Merchant Django-Merchant is a django application that enables you to use multiple payment processors from a single API. Gateways Following gate

Agiliq 997 Dec 24, 2022
Django friendly finite state machine support

Django friendly finite state machine support django-fsm adds simple declarative state management for django models. If you need parallel task executio

Viewflow 2.1k Dec 31, 2022
A Minimalistic Modern Django Boilerplate

A Minimalistic Modern Django Boilerplate This boilerplate is mainly for educational purposes. It is meant to be cloned as a starter code for future tu

Jonathan Adly 21 Nov 02, 2022
Django/Jinja template indenter

DjHTML A pure-Python Django/Jinja template indenter without dependencies. DjHTML is a fully automatic template indenter that works with mixed HTML/CSS

Return to the Source 378 Jan 01, 2023
django+bootstrap5 实现的 个人博客

项目状态: 正在开发中【目前已基本可用】 项目地址: https://github.com/find456789/django_blog django_blog django+bootstrap5 实现的 个人博客 特点 文章的历史版本管理(随时回退) rss、atom markdown 评论功能

名字 3 Nov 16, 2021
Djang Referral System

Djang Referral System About | Features | Technologies | Requirements | Starting | License | Author 🎯 About I created django referral system and I wan

Alex Kotov 5 Oct 25, 2022
Inject an ID into every log message from a Django request. ASGI compatible, integrates with Sentry, and works with Celery

Django GUID Now with ASGI support! Django GUID attaches a unique correlation ID/request ID to all your log outputs for every request. In other words,

snok 300 Dec 29, 2022
This a Django TODO app project and practiced how to deploy and publish the project to Heroku

ToDo App Demo | Project Table of Contents Overview Built With Features How to use Acknowledgements Contact Overview Built With HTML CSS JS Django How

Cetin OGUT 1 Nov 19, 2021
Python port of Google's libphonenumber

phonenumbers Python Library This is a Python port of Google's libphonenumber library It supports Python 2.5-2.7 and Python 3.x (in the same codebase,

David Drysdale 3.1k Jan 08, 2023
A simple polling app made in Django and Bootstrap

DjangoPolls A Simple Polling app made with Django Instructions Make sure you have Python installed Step 1. Open a terminal Step 2. Paste the given cod

Aditya Priyadarshi 1 Nov 10, 2021
Customize the behavior of django.contrib.auth permissions.

Customizando o comportamento do django.contrib.auth. O que queremos? Não criar as permissões padrões automaticamente (add, delete, view, read). Criar

Henrique Bastos 7 Nov 26, 2022
Streamlining Django forms to provide all the wins of single-page-applications without the pain.

nango Streamlining Django forms to provide all the wins of single-page-applications without the pain. Key features Available to all Django deployments

Nick Farrell 107 Dec 12, 2022