Django application and library for importing and exporting data with admin integration.

Overview

django-import-export

Build status on Travis-CI https://coveralls.io/repos/github/django-import-export/django-import-export/badge.svg?branch=master Current version on PyPi Documentation

PyPI - Python Version

PyPI - Django Version

django-import-export is a Django application and library for importing and exporting data with included admin integration.

Features:

  • support multiple formats (Excel, CSV, JSON, ... and everything else that tablib support)
  • admin integration for importing
  • preview import changes
  • admin integration for exporting
  • export data respecting admin filters

docs/_static/images/django-import-export-change.png

Example app

To run the demo app:

cd tests
./manage.py makemigrations
./manage.py migrate
./manage.py createsuperuser
./manage.py loaddata category book
./manage.py runserver

Contribute

If you'd like to contribute, simply fork the repository, commit your changes to the develop branch (or branch off of it), and send a pull request. Make sure you add yourself to AUTHORS.

As most projects, we try to follow PEP8 as closely as possible. Please bear in mind that most pull requests will be rejected without proper unit testing.

Comments
  • Better surfacing of validation errors in UI / optional model instance validation

    Better surfacing of validation errors in UI / optional model instance validation

    Hi @bmihelac,

    Related issues:

    • #64
    • #176
    • #415
    • #601

    Here's where I've got to so far. I thought it would be good to get some feedback before getting stuck into writing new tests, but the current ones are passing, so at least nothing is broken :)

    When enabled, instance.full_clean() is called for each row, and any resulting validation errors surface themselves in the preview table like so:

    screenshot 2018-11-02 at 15 35 37

    The errors are hidden visually at first, then become visible when you hover over a row's first column, in such a way that it doesn't hide any of the values for the current row. It's pretty crude, but it's a starting point. Some examples:

    screenshot 2018-11-02 at 15 35 42 screenshot 2018-11-02 at 15 35 53

    Anyone wanting to give this a whirl for themselves should be able to install from git using:

    pip install git+https://github.com/ababic/[email protected]

    Any/all feedback welcome :)

    opened by ababic 43
  • when import csv file ,KeyError: u'id' occured

    when import csv file ,KeyError: u'id' occured

    i try to import some data form csv file, but errors occurred as follows:

    Errors Line number: 1 - u'id' Traceback (most recent call last): File "/home/zhijiasun/.virtualenvs/tutorial/local/lib/python2.7/site-packages/import_export/resources.py", line 317, in import_data instance, new = self.get_or_init_instance(instance_loader, row) File "/home/zhijiasun/.virtualenvs/tutorial/local/lib/python2.7/site-packages/import_export/resources.py", line 149, in get_or_init_instance instance = self.get_instance(instance_loader, row) KeyError: u'id'

    I just modify the data of the csv file which exported by the app and then try to import it. But error happed.

    stale 
    opened by zhijiasun 36
  • question:UnicodeDecodeError at /admin/blog/author/import/

    question:UnicodeDecodeError at /admin/blog/author/import/

    Sir,thanks a lot to develop this great repository. I have little problem to solve , as below the code:

    #in models.py
    class Author(models.Model):
        author = models.CharField('作者', max_length=50)
        title = models.CharField('标题', max_length=150)
        qualification = models.ForeignKey(Qualification)
        mark = models.ManyToManyField(Mark)
        blog = models.TextField('博客内容')
        time = models.DateField('写作日期')
    
        def __unicode__(self):
            return unicode(self.author)
    
        class Meta:
            ordering = ['time']
    
    #in admin.py
    class AuthorAdmin(ImportExportModelAdmin, admin.ModelAdmin):
        search_fields = ('author', 'title', 'mark', 'blog')
        list_display = ('author', 'title', 'time')
    

    When I export the data, 1 but the csv file appear to be the retortion, 2 when I modified the second row author data and import,

    3 it cause the error, 4

    how I it be smoothly modified and import?thanks. Allenlee

    opened by hebijiandai 31
  • CLI import (reprise)

    CLI import (reprise)

    This is a continuation of @int-ua's work in #725, addressing #332. I've resolved conflicts, tweaked the documentation and tests, but otherwise kept it identical.

    I need this feature too, so I'd like to get it merged sooner rather than later. However, there are other tweaks I'd like to make, including:

    • Better documentation, including its own page in the Reference
    • Format options:
      • arg to list available formats, calling into base_formats rather than being hardcoded
      • arg to specify format to use rather than guessing
    • Detect whether supplied class is a Resource or Model rather than needing to specify
    • raise_errors currently defaults to True if dry_run is False - is this the right thing to do?
    • Option to supply kwargs to the Resource constructor (which I need for my own project)
    • Hey, export would be nice too

    ... but let's start with this, and if I get time for those tweaks I'll make them in a separate PR.

    enhancement needs update 
    opened by yozlet 27
  • Added ExportViewMixin

    Added ExportViewMixin

    Created a mixin view to export without admin interface.

    I think it would be a really great thing if we have a generic class based views for this application.

    I tested this patch with a ListView & django_filters.views.FilterView. The 2nd is really useful to replace the AdminFilters

    needs update 
    opened by ZuluPro 23
  • Many to many fields and admin upload

    Many to many fields and admin upload

    Do many to many fields work via admin imports? I keep getting something like this: ValueError('"<Accession: >" needs to have a value for field "accession" before this many-to-many relationship can be used.',)

    It doesn't seem like it can if it's displaying the data to import before importing, as there's no way to get the primary key for the row since it doesn't have one yet?

    Thanks,

    Dean

    opened by lfarrell 23
  • Refactor form-related methods on ImportMixin

    Refactor form-related methods on ImportMixin

    Problem

    Overriding the forms for the import process is currently more difficult than it could be, and there's a bit of 'difficult to follow' stuff going on in the current implementation - particularly in how get_form_kwargs() is used to create the confirmation form.

    Solution

    First of all, new import_form_class and confirm_form_class attributes allow you to easily specify custom form classes for the import process without having to override methods.

    The current get_import_form() and get_import_confirm_form() methods have been deprecated in favour of using the more descriptively named get_import_form_class() and get_confirm_form_class() methods.

    The single get_form_kwargs() method has been deprecated in favour of adding separate methods for each form:

    • get_import_form_kwargs(request, form_class, **kwargs)
    • get_confirm_form_kwargs(request, form_class, import_form=None, **kwargs)

    And to more clearly separate the logic for customising 'intial data' and 'init kwargs', there are separate dedicated methods for each form:

    • get_import_form_initial(request, form_class, **kwargs)
    • get_confirm_form_initial(request, form_class, import_form=None, **kwargs)

    Last but not least, the entire business of form instantiation has been factored out into two new methods:

    • create_import_form(request, **kwargs)
    • create_confirm_form(request, import_form=None, **kwargs)

    These should make it super convenient for modifying forms on the fly.

    Paired with the changes from #1038, this should make for a much more developer-friendly implementation.

    To do

    • [x] Update documentation to reflect changes
    • [x] Write tests to cover new behaviour
    opened by ababic 22
  • datetime field localtime() issue

    datetime field localtime() issue

    Describe the bug Django DateTime Import

    To Reproduce Import DateTime data and you'll hit a "localtime() cannot be applied to a naive datetime", I've been using the package for a while now and use the admin import page as well as a custom import page for users. Both work fine for non-datetime data.

    I've tried this with adding with tz +00 and without 2020-07-17 7:42:39 2020-07-17 7:42:39+00

    I've tried the DateTimeWidget where format="%Y-%m-%d %H:%M:%S" format="%Y-%m-%d %H:%M:%S$z" format=None

    I've posted a SOF question regarding it https://stackoverflow.com/questions/62990575/django-import-export-datetime-naive-datetime-error

    And had a convo specific to it on SOF https://chat.stackoverflow.com/rooms/218260/django-import-export

    The end result seems that the issue may be in the django-import-export package.

    Versions (please complete the following information):

    • django-import-export==2.2.0
    • Python 3.7.4
    • Django==3.0.2

    settings.py

    LANGUAGE_CODE = 'en-us'
    TIME_ZONE = 'UTC'
    USE_I18N = True
    USE_L10N = True
    USE_TZ = True
    

    Expected behavior I was expecting the datetime to import the dates in my csv file

    Screenshots View 2 links above

    Additional context View 2 links above

    bug 
    opened by MrAlexWinkler 21
  • 1.8 compatibility

    1.8 compatibility

      File "/Users/syabro/.envs/homeowners/lib/python2.7/site-packages/import_export/resources.py", line 16, in <module>
        from django.db.models.related import RelatedObject
    ImportError: No module named related
    

    because

    The django.db.models.related module has been removed and the Field.related attribute will be removed in Django 2.0.

    https://docs.djangoproject.com/en/dev/releases/1.8/#model-field-related

    bug 
    opened by syabro 19
  • During import, table id will not be auto-incremented

    During import, table id will not be auto-incremented

    Running on:

    • Django-1.6
    • django-import-export-0.1.5
    • postgresql.x86_64-9.2.5-1
    • tablib-0.9.11
    • python-2.7.5-9

    I have successfully imported data from a csv file.

    When I try to add more data manually, I receive the error:

    duplicate key value violates unique constraint "mytable_pkey"
    DETAIL:  Key (id)=(1) already exists.
    

    From psql:

    select max(id) from mytable;
      75
    
    SELECT nextval('mytable_id_seq');
      2
    

    Does django-import-export support the table's id auto-incrementing?

    opened by raratiru 18
  • Pass row values to every Widget

    Pass row values to every Widget

    AFAIU it will help create widgets with autocreation of related ForeignKey instances that require two fields already present in the row. For example:

    • FK to Person with first_name, last_name
    • FK to Quote that requires a Book (let's suppose it's identifiable just by name) and that book is already in the import fields
    • FK to POI that is identifiable with two (or even three) coordinates
    enhancement 
    opened by int-ua 17
  • added select_ralted option when exporting model resoureces

    added select_ralted option when exporting model resoureces

    Problem

    ovveriding get_queryset not working when exporting from admin panel

    Solution

    How did you solve the problem?

    add export_selected_related option to include when making the queryset

    usage:

    class MyResource(resources.ModelResource):
        export_selected_related = ('RELATED_MODEL1', 'RELATED_MODEL2', ...)
    

    Acceptance Criteria

    Have you written tests? Have you included screenshots of your changes if applicable? Did you document your changes?

    opened by yusufadell 2
  • dry_run kwarg not passed to all methods

    dry_run kwarg not passed to all methods

    dry_run is passed into import_row() but it is hit and miss as to whether this param (and other named kwargs) are passed to subsequent methods such as after_import_instance().

    It would help users (e.g. this example) if these flags were available to other methods. I would propose that they are passed in kwargs rather than as named params, so as not to clutter the method definitions.

    bug enhancement 
    opened by matthewhegarty 0
  • 'list' object is not callable

    'list' object is not callable

    Describe the bug I tried to impliment admin class for Books table along with ImportExportModelAdmin and ImportExportActionModelAdmin. while exporting selected records from admin panel i got the error.

    To Reproduce Steps to reproduce the behavior:

    1. Go to admin.py and impliment custom model admin along with ImportExportModelAdmin and ImportExportActionModelAdmin
    2. Got to respective table in admin panel. Select few records and export them as csv file
    3. See error
    class Base(models.Model):
        slug = models.SlugField(max_length = 200)
    
        class Meta:
            abstract = True
    
    
    class Books(Base):
        name = models.CharField('Book name', max_length=100)
        author = models.ForeignKey(Author, blank=True, null=True)
        author_email = models.EmailField('Author email', max_length=75, blank=True)
        imported = models.BooleanField(default=False)
        published = models.DateField('Published', blank=True, null=True)
        price = models.DecimalField(max_digits=10, decimal_places=2, null=True, blank=True)
        categories = models.ManyToManyField(Category, blank=True)
    
        def __str__(self):
            return self.name
    
    
    class BaseAdmin(admin.ModelAdmin):
        list_display = ('slug', )
    
    
    class BooksResource(ModelResource):
        class Meta:
            model = Books
    
    
    class BooksAdmin(ImportExportModelAdmin, ImportExportActionModelAdmin, BaseAdmin):
        resource_class = [BooksResource]
        list_display = BaseAdmin.list_display + ('name', 'author',  'published', 'price', 'categories', )
    
    
    admin.site.register(Books, BooksAdmin)
    

    Versions (please complete the following information):

    • Django Import Export: [e.g. 3.0.1]
    • Python [e.g. 3.9]
    • Django [e.g. 4.1]

    Expected behavior A clear and concise description of what you expected to happen.

    Screenshots If applicable, add screenshots to help explain your problem.

    Additional context Add any other context about the problem here.

    bug 
    opened by kumar-student 2
  • Rename `after_import_instance()` method for clarity

    Rename `after_import_instance()` method for clarity

    The after_import_instance() is a misnomer because it is called after the instances is loaded (or created) and not after it is imported. Existing clients rely on this method so it would have to be deprecated correctly rather than simply renaming.

    It would also be a good idea to pass the row in as a kwarg so that that users can use row data if they need to.

    enhancement good first time issue 
    opened by matthewhegarty 0
  • handle use case where relations need to be created prior to import

    handle use case where relations need to be created prior to import

    If a user needs to create related objects prior to import and those objects are identified by a combination of fields, then there isn't a 'clean' way of doing this. When the relation is identified by a single field, then we can override ForeignKeyWidget for this, or create data in before_import_row(). The import will skip any fields which are not present in the row, hence using an overridden ForeignKeyWidget won't work, but perhaps using ForeignKeyWidget in some way would be the best solution. It would have to be called if the field was not present in the row data.

    In this related SO issue, I suggest overriding import_obj(), but it would be cleaner to override before_save_instance(). This could be done if the 'row' data is passed to before_save_instance().

    bug 
    opened by matthewhegarty 0
  • `import_obj()` declaration can be improved for consistency

    `import_obj()` declaration can be improved for consistency

    Throughout the code base, the param name row is used to define the incoming row (even though this could be JSON or YAML). In import_obj() it is named data, which is less descriptive. We should standardise variable naming for better readability.

    enhancement good first time issue 
    opened by matthewhegarty 0
Releases(2.8.0)
Django-shared-app-isolated-databases-example - Django - Shared App & Isolated Databases

Django - Shared App & Isolated Databases An app that demonstrates the implementa

Ajai Danial 5 Jun 27, 2022
django-reversion is an extension to the Django web framework that provides version control for model instances.

django-reversion django-reversion is an extension to the Django web framework that provides version control for model instances. Requirements Python 3

Dave Hall 2.8k Jan 02, 2023
The little ASGI framework that shines. 🌟

✨ The little ASGI framework that shines. ✨ Documentation: https://www.starlette.io/ Community: https://discuss.encode.io/c/starlette Starlette Starlet

Encode 7.7k Dec 31, 2022
Automated image processing for Django. Currently v4.0

ImageKit is a Django app for processing images. Need a thumbnail? A black-and-white version of a user-uploaded image? ImageKit will make them for you.

Matthew Dapena-Tretter 2.1k Dec 17, 2022
PEP-484 stubs for django-rest-framework

pep484 stubs for Django REST framework Mypy stubs for DRF 3.12.x. Supports Python 3.6, 3.7, 3.8 and 3.9. Installation pip install djangorestframework-

TypedDjango 303 Dec 27, 2022
Let AngularJS play well with Django

django-angular Let Django play well with AngularJS What does it offer? Add AngularJS directives to Django Forms. This allows to handle client side for

Jacob Rief 1.2k Dec 27, 2022
Exploit Discord's cache system to remote upload payloads on Discord users machines

Exploit Discord's cache system to hide payloads PoC Remote upload embedded payload from image using EOF to Discord users machines through cache. Depen

cs 169 Dec 20, 2022
mirage ~ ♪ extended django admin or manage.py command.

mirage ~ ♪ extended django admin or manage.py command. ⬇️ Installation Installing Mirage with Pipenv is recommended. pipenv install -d mirage-django-l

Shota Shimazu 6 Feb 14, 2022
This is django-import-export module that exports data into many formats

django-import-export This is django-import-export module which exports data into many formats, you can implement this in your admin panel. - Dehydrat

Shivam Rohilla 3 Jun 03, 2021
Run Django tests with testcontainers.

django-rdtwt (Run Django Tests With Testcontainers) This targets users who wish to forget setting up a database for tests. There's no manually startin

2 Jan 09, 2022
REST API con Python, Django y MySQL (GET, POST, PUT, DELETE)

django_api_mysql REST API con Python, Django y MySQL (GET, POST, PUT, DELETE) REST API con Python, Django y MySQL (GET, POST, PUT, DELETE)

Andrew 1 Dec 28, 2021
A Django based shop system

django-SHOP Django-SHOP aims to be a the easy, fun and fast e-commerce counterpart to django-CMS. Here you can find the full documentation for django-

Awesto 2.9k Dec 30, 2022
Simple Login Logout System using Django, JavaScript and ajax.

Djanog-UserAuthenticationSystem Technology Use #version Python 3.9.5 Django 3.2.7 JavaScript --- Ajax Validation --- Login and Logout Functionality, A

Bhaskar Mahor 3 Mar 26, 2022
A ToDO Rest API using Django, PostgreSQL and Docker

This Rest API uses PostgreSQL, Docker and Django to implements a ToDo application.

Brenno Lima dos Santos 2 Jan 05, 2022
Declarative model lifecycle hooks, an alternative to Signals.

Django Lifecycle Hooks This project provides a @hook decorator as well as a base model and mixin to add lifecycle hooks to your Django models. Django'

Robert Singer 1k Dec 31, 2022
Zendesk Assignment - Django Based Ticket Viewer

Zendesk-Coding-Challenge Django Based Ticket Viewer The assignment has been made using Django. Important methods have been scripted in views.py. Excep

Akash Sampurnanand Pandey 0 Dec 23, 2021
Simple tagging for django

django-taggit This is a Jazzband project. By contributing you agree to abide by the Contributor Code of Conduct and follow the guidelines. django-tagg

Jazzband 3k Jan 02, 2023
Django models and endpoints for working with large images -- tile serving

Django Large Image Models and endpoints for working with large images in Django -- specifically geared towards geospatial tile serving. DISCLAIMER: th

Resonant GeoData 42 Dec 17, 2022
Django Livre Bank

Django Livre Bank Projeto final da academia Construdelas. API de um banco fictício com clientes, contas e transações. Integrantes da equipe Bárbara Sa

Cecília Costa 3 Dec 22, 2021
A helper for organizing Django project settings by relying on well established programming patterns.

django-configurations django-configurations eases Django project configuration by relying on the composability of Python classes. It extends the notio

Jazzband 953 Dec 29, 2022