Send e-mails asyncronously using cron

Overview

django-yubin

https://codecov.io/github/APSL/django-yubin/coverage.svg?branch=master Documentation Status

Django Yubin allows the programmer to control when he wants to send the e-mail in this application, making the web application to answer fast as it not has to wait for the mail server.

As in our projects we use always two django packages for dealing with emails: django-mailer2 (our own fork in APSL) and django-mailviews to compose the emails we decided to create this package to fit our own needs and share with the community.

As you can see it seems django-mailer2 is not accepting patches, so in order to put a new version on pypi a new name was mandatory. So django-yubin was born (yubin is postal mail in japanese). The name attribution is for @morenosan.

How it works

Yubin replaces the standard Django Email Backend with its own. Instead of sending the e-mail trough the SMTP server Yubin stores the e-mails on the database and allows you to sent them using a cron command.

Advantages

  • Your application can answer faster, as other process is going to take care of connecting with the SMTP server and sending the e-mail.
  • Yubin stores the e-mail and allows you to retrieve using the admin. Even with the attachments.
  • Yubin allows to define prioritary queues, resend e-mails
  • Yubin helps in your development. It's a good way to work when you're developping the application and you don't want to flood your test users with e-mails. With Django Yubin, and without running the cron commands, you can see how your e-mails are, retrieve them and even delete them with out sending it.

On production mode you'll just nedd to add a cron entry in your server to send the e-mails, someting like

          • (cd $PROJECT; python manage.py send_mail >> $PROJECT/cron_mail.log 2>&1)

This will send the queued e-mail every minute.

Django Yubin is a fork from django-mailer2 with some addtions from django-mailviews and additional improvements made from apsl.net that we need for our daly basis workd. It has also contributions from other people, so don't heasitate to read the humans.txt.

django-mailer-2 by is a Chris Beaven fork from a fork of James Tauber's django-mailer and is a reusable Django app for queuing the sending of email.

django-mailviews from Disqus, allows you to compose e-mails using templates in the same way you compose your html templates, and allows you to preview the e-mails.

If you want to run the test you'll need a test smtpd server, you can find one in

./bin/fake-server

run it in a different console and execute runtests.py

You can read the package documentation at http://django-yubin.readthedocs.org/en/latest/

Changelog

  • 1.7.0 Add optional MAILER_MESSAGE_SEARCH_FIELDS setting. It's a tuple of strings with the fields to use in admin.Message.search_fields attribute.
  • 1.6.0 Support for Django 3.0
  • 1.5.0 New TemplatedMultipleAttachmentsEmailMessageView to allow to send emails with more than 1 attachment.
  • 1.4.1 Detecting if messages are encoding using different encoding headers to be able to preview them (now base64, quoted-printable).
  • 1.4.0 Option added in status_mail command to return the output in json format.
  • 1.3.1 Fix unicode and encode errors: sending queued and non queued emails and in admin detail view.
  • 1.3.0 Allow to send emails inmediatly without being saved in database (priority «now-not-queued»). Add support for Python 3.7 and Django 2.1. Remove old code for Django < 1.3.
  • 1.2.0 Fix is_base64 detection. Add a «send_test_email» command to check connection parameters. New health check view. Don't open a connection if there are no messages in queue to send. Add a "date_sent" field to detect when the mail was sent.
  • 1.1.0 Fix attachment headers in TemplateAttachmentEmailMessagView making both "attachment" and "filename" args mandatory.
  • 1.0.5 Add missing paths in MANIFEST.in.
  • 1.0.4 Fix attachment visualization in the admin. Attach pdf in create_mail command. Solved Content-Transfer-Encoding issue.
  • 1.0.3 Fixed issue decoding the message payload, added support for django 1.9, updated changelog and added support to deploy the package from travis.
  • 1.0.0 Add support for Django 2.0 and remove django 1.8.
  • 0.8.2 Fix date created column in QueuedMessages admin.
  • 0.8.1 Ensure that LOCK_WAIT_TIMEOUT is never negative to avoid a bug in lockfile in systems which use a LinkFileLock.
  • 0.8.0 Use settings.MAILER_PAUSE_SEND to skip smtp connections. Fix UTF-8 encoding in messages. Fix encoding errors in email visualization in the admin.
  • 0.7.0 Fix template context bug for Django 1.11. Add Python 3.6 to CI and drop Python 3.3 and Django 1.9.
  • 0.6.0 Support for Python 3.6.
  • 0.5.0 Limit nº of emails sent by send_mail command. Update the debug handlers options for verbosity to accept v3.
  • 0.4.0 Support Django 1.11: subject and body are no longer unscaped, you need to add {% autoescape off %} to your non HTML templates.
  • 0.3.1 Delete unused template that caused an error with django-compressor offline. testmail command now generates HTML emails.
  • 0.3.0 Support Django >= 1.8 and <=1.10, Python 2.7, 3.3, 3.4 and 3.5. Re-send mails admin action. Fix bug in status_mail command. Demo project configured to send mails with the same mail fake-server used for tests.
  • 0.2.3 Removed {% load url from future %} to support Django 1.9. Now Django < 1.5 is not supported.
  • 0.2.2 Include migrations directory in .tar.gz in PyPi.
  • 0.2.1 Updated links to CI and Code Coverage Services
  • 0.2.0 Merged sergei-maertens contribution.
  • 0.1.8 Added migrations for Django 1.9 compatibility. See http://django-yubin.readthedocs.org/en/latest/install.html#upgrading-from-previous-versions
  • 0.1.7 Support for Django 1.8.
  • 0.1.6 Bugfixes.
  • 0.1.5 Bugfixes.
  • 0.1.4 Updated README.
  • 0.1.3 Fixed Python3 compatibility, thanks Marc, Cesc & Dani.
  • 0.1.2 Fixed Templates.
  • 0.1.1 Updated documentation and unit tests.
Comments
  • Limit Emails per Call

    Limit Emails per Call

    To avoid mail throttles/spamming this adds a message_limit option to limit how many emails can get sent in a single call. I tested this with a number of settings and it's working for me.

    opened by the-mace 15
  • Introduce additional permission checks

    Introduce additional permission checks

    In the custom views in the Message admin check check if the 'change' permission for the 'Message' model is set in addition to the existing check if for is_active and is_staff which the django's admin_site.admin_view decorator introduces.

    opened by flupzor 8
  • Only warn if Yubin deferred or skipped mail

    Only warn if Yubin deferred or skipped mail

    It might be a matter of preference, however using loglevel warning for regular behaviour (1 or more emails have been sent successfully) seems incorrect in my eyes.

    As such a slight tweak to only use warning when Yubin defers or skips 1 or more emails.

    opened by alextreme 7
  • (fix) UnicodeEncodeError when priority is 'now-not-queued'

    (fix) UnicodeEncodeError when priority is 'now-not-queued'

    When trying to send email with priority 'now-not-queued' I can see in the errors files messages as :

    failed due to: 'ascii' codec can't encode character '\xf3' in position 481: ordinal not in range(128)
    

    This is the summarize:

    • Add testUnicodePriorityNowNotQueuedMessage
    • Bug fix for UnicodeEncodeError when priority is 'now-not-queued'
    • Bug fix for counting num_sent emailsI
    • Change models.QueuedMessage.objects.all() -> models.QueuedMessage.objects.filter(deferred__isnull=True) to check there is not deferred
    opened by xusy2k 6
  • Unicode encode errors

    Unicode encode errors

    Fix UnicodeEncodeErrors:

    • Sending queued emails
    • Sending non queued emails
    • In admin detail view

    Thanks to @xusy2k to find an trying to fix the bugs in PR #48.

    opened by sastred 5
  • Lock already in place. Exiting.

    Lock already in place. Exiting.

    Im having a significant issue with the lock system.

    Usually yubin works great for everything. However, sometimes the lock file does not get deleted properly when the program exits. I dont know if its because of abnormal termination or due to some other reason.

    Whatever the cause - after that, yubin cant run anymore because it sees the old lock file. Since Im not getting emails, sometimes it takes me days or weeks to realize that there is a problem. This happens rarely but often enough - roughly every few months.

    Obviously this is a really serious issue on a production system.

    I see there is a finally: block where the lock is supposed to get released. I have no idea why its not getting there sometimes.

    Im not so familiar with the system, so the only solution I can think of is to create a management command that can be run say once or twice a day that checks for a really old lock file and deletes it. For instance a lock file thats 4 hours old is probably not really supposed to be there.

    Maybe there is a more elegant and correct solution but thats all I can come up with given my limited knowledge

    Thanks

    Daniel

    opened by drownload 4
  • UnicodeDecodeError

    UnicodeDecodeError

    Traceback (most recent call last): File "./manage.py", line 10, in execute_from_command_line(sys.argv) File "/home/healersource/lib/python2.7/django/core/management/init.py", line 354, in execute_from_command_line utility.execute() File "/home/healersource/lib/python2.7/django/core/management/init.py", line 346, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/healersource/lib/python2.7/django/core/management/base.py", line 394, in run_from_argv self.execute(*args, **cmd_options) File "/home/healersource/lib/python2.7/django/core/management/base.py", line 445, in execute output = self.handle(*args, **options) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/management/commands/send_mail.py", line 56, in handle message_limit=options['message_limit']) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/engine.py", line 113, in send_all blacklist=blacklist) File "/home/healersource/webapps/hs/env/lib/python2.7/site-packages/django_yubin/engine.py", line 207, in send_queued_message smart_str(message.encoded_message).encode('utf-8')) UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 559: ordinal not in range(128)

    opened by drownload 4
  • No CI for pushes/pull requests (Travis)

    No CI for pushes/pull requests (Travis)

    I see that there is no Travis integration. I could add this (set up the whole tox stack) so you can at least see if the build is failing for certain Python/Django versions or not.

    What's the support policy? Which Python/Django versions? I'd suggest Py27, Py34 and Py35, Django 1.8 and Django 1.9. For the rest I would point to django-mailer-2, which is unsupported but so are the Python/Django versions lower than those specified.

    opened by sergei-maertens 4
  • TemplateAttachmentEmailMessagView: filename required if attachment is provided

    TemplateAttachmentEmailMessagView: filename required if attachment is provided

    Before this fix, when an attachment was added using the render_to_message the filename and the content of the attachment wasn't provided to the attach method at the same time. This provoke that the Content-Disposition header wasn't added at the attachment multipart.

    The proposed solution force to assign a filename when an attachment is added (and vicecersa), in order to provide all the parameters to the attach method in order that the Content-Disposition header is written inside the attachment multipart.

    opened by csalom 3
  • Update the debug handlers options for verbosity

    Update the debug handlers options for verbosity

    Since the commands (send_mail etc.) are using the standard -v verbosity in the arg parser, the help for the command shows that there are four verbosity levels (0-3). At the moment if you run

    python manage.py send_mail -v3

    for instance, it fails with a cryptic message about the key "3" not being present in the LOGGING_LEVEL dictionary. This change duplicates the "2" key so that both 2 and 3 verbosity have the DEBUG message level.

    An alternative would be to specify the -v option in each of the commands to only allow -v[0/1/2].

    opened by davepeake 3
  • Rate limiting

    Rate limiting

    I've been using django-yubin for a while and it's a great addition. My challenge is sometimes my queue gets too many items in it and then I have a large amount of emails to send (all opt-in, no spamming). My email provider doesn't like it when the volume spikes and starts aggressively throttling and can even lock the account for a while. It would be really helpful if django-yubin had some way of limiting the number of emails per time period. So feed from the queue but if we've sent X emails in this cron cycle then stop sending emails until next cycle.

    Is anything like this possible now or is it possible to add something like this?

    I found an option, block_size, in the code which I didn't see in the docs. It's related but still sends all mail in the queue.

    opened by the-mace 3
  • To support creating a single Email object with several addresses

    To support creating a single Email object with several addresses

    It would be nice to support multiple addresses (either with to / cc / bcc fields) for the same Email. Currently, a new Message object is created per address.

    By allowing the same email to be sent to multiple addresses, we could reduce table space significantly.

    enhancement 
    opened by celdrake 0
Releases(1.7.1)
Owner
APSL
Web, mobile apps and DevOps mostly in Python, Django, JavaScript and Go.
APSL
Esio_dev 3 Oct 15, 2021
Generate Email, Register for anything, Get the OTP/Link

OTE : One Time Email Introduction ote is a command line utility that generates temporary email address and automatically extracts OTPs or confirmation

Somdev Sangwan 457 Jan 03, 2023
A Django app that allows you to send email asynchronously in Django. Supports HTML email, database backed templates and logging.

Django Post Office Django Post Office is a simple app to send and manage your emails in Django. Some awesome features are: Allows you to send email as

User Inspired 856 Dec 25, 2022
An email sending system with random confirmation code.

email_sending An email sending system with random confirmation code. Description Confirmation emails are sent based on the list of email addresses. Ea

Larissa Queiroz 2 Mar 22, 2022
Kanmail - An email client that functions like a kanban board, for Mac/Windows/Docker

Kanmail - An email client that functions like a kanban board, for Mac/Windows/Docker

Oxygem 1.2k Dec 31, 2022
An offline Phishing Email Analyzer.

An offline Phishing Email Analyzer.

Kamran Saifullah (Frog Man) 11 Oct 19, 2022
Fastapi mail system sending mails(individual, bulk) attachments(individual, bulk)

Fastapi-mail The fastapi-mail simple lightweight mail system, sending emails and attachments(individual && bulk) 🔨 Installation $ pip install fastap

Sabuhi 399 Dec 29, 2022
📧 CLI to deduplicate mails from mail boxes.

Mail Deduplicate Command-line tool to deduplicate mails from a set of boxes. Stable release: Development: Features Duplicate detection based on cherry

Kevin Deldycke 134 Dec 14, 2022
A Python Mail Server

Salmon - A Python Mail Server Download: https://pypi.org/project/salmon-mail/ Source: https://github.com/moggers87/salmon Docs: https://salmon-mail.re

Matt Molyneaux 582 Dec 30, 2022
A light-weight, modular, message representation and mail delivery framework for Python.

Marrow Mailer A highly efficient and modular mail delivery framework for Python 2.6+ and 3.2+, formerly called TurboMail. © 2006-2019, Alice Bevan-McG

Marrow Open Source Collective 255 Dec 28, 2022
Suplantar mails de empresas como google, facebook, github, etc...

Suplantar mails de empresas como google, facebook, github, etc...

piter 3 Feb 05, 2022
Python IMAP for Human beings

Imbox - Python IMAP for Humans Python library for reading IMAP mailboxes and converting email content to machine readable data Requirements Python (3.

Martin Rusev 1.1k Dec 30, 2022
Yahoo Mail Validator For Python

Validator Validator helps to know if the mail is valid or not Installation Install The libraries pip install requests bs4 colorama Usage Create a new

Mr Python 3 Mar 12, 2022
Euserv_extend captcha solver + pin code(Gmail)

Euserv_extend captcha solver + pin code(Gmail)

19 Nov 30, 2022
Python email address and Mime parsing library

Flanker - email address and MIME parsing for Python Flanker is an open source parsing library written in Python by the Mailgun Team. Flanker currently

Mailgun Team 1.6k Dec 29, 2022
An automation program that checks whether email addresses are real, whether they exist and whether they are a validated mail

Email Validator It is an automation program that checks whether email addresses are real, whether they exist and whether they are a validated mail. Re

Ender MIRIZ 4 Dec 22, 2021
A django package which act as a gateway to send and receive email with amazon SES.

django-email-gateway: Introduction: A Simple Django app to easily send emails, receive inbound emails from users with different email vendors like AWS

MicroPyramid 28 Nov 09, 2022
A script based on an article I wrote on decluttering emails.

Decluttering_Email A script based on an article I wrote on decluttering emails. What does this program do? This program is a python script that sends

Ogheneyoma Obomate Okobiah 6 Oct 21, 2021
A SMTP server for use as a pytest fixture that implements encryption and authentication.

SMTPDFix: Test email, locally A simple SMTP server based on aiosmtpd for use as a fixture with pytest that supports encryption and authentication. All

James Warne 11 Sep 03, 2022
A Pythonic interface for Google Mail

GMail for Python A Pythonic interface to Google's GMail, with all the tools you'll need. Search, read and send multipart emails, archive, mark as read

Charlie Guo 1.7k Dec 29, 2022