Python email address and Mime parsing library

Related tags

Emailflanker
Overview

Flanker - email address and MIME parsing for Python

https://travis-ci.org/mailgun/flanker.svg?branch=master https://coveralls.io/repos/github/mailgun/flanker/badge.svg?branch=master

Flanker is an open source parsing library written in Python by the Mailgun Team. Flanker currently consists of an address parsing library (flanker.addresslib) as well as a MIME parsing library (flanker.mime).

Detailed documentation is provided in the User Manual as well as the API Reference. A Quickstart Guide is provided below.

Python Versions

Flanker is heavily used by Mailgun in production with Python 2.7. The current production version is v0.8.5.

Support for Python 3 was added in v0.9.0 by popular demand from the community. We are not using Flanker with Python 3 in the house. All we know is that tests pass with Python 3.6, so use at your own risk. Feel free to report Python 3 specific issues if you see any.

Installing

You can install flanker via pip or clone the repo from GitHub.

You'll need Python headers files before you start working with flanker, so install them first:

# ubuntu
sudo apt-get install python-dev
# fedora
sudo yum install python-devel

If you are using pip, simply type:

pip install flanker

If you are cloning from GitHub, you can type:

git clone [email protected]:mailgun/flanker.git
cd flanker
pip install -e .

Address Parsing

To parse a single mailbox (display name as well as email address):

>>> from flanker.addresslib import address
>>>
>>> address.parse('Foo [email protected]')
Foo <foo@example.com>

An invalid address is returned as None:

>>> from flanker.addresslib import address
>>>
>>> print address.parse('@example.com')
None

To parse a single email address (no display name):

>>> from flanker.addresslib import address
>>>
>>> address.parse('[email protected]', addr_spec_only=True)
foo@example.com

To parse an address list:

>>> from flanker.addresslib import address
>>>
>>> address.parse_list(['[email protected], [email protected], @example.com'])
[foo@example.com, bar@example.com]

To parse an address list as well as return a tuple containing the parsed addresses and the unparsable portions

>>> from flanker.addresslib import address
>>>
>>> address.parse_list(['[email protected], [email protected], @example.com'], as_tuple=True)
[foo@example.com, bar@example.com], ['@example.com']

To parse an address list in strict mode:

>>> from flanker.addresslib import address
>>>
>>> address.parse_list(['[email protected], [email protected], @example.com'], strict=True)
[foo@example.com, bar@example.com]

To validate an email address (parse as well as DNS, MX existence, and ESP grammar checks):

>>> from flanker.addresslib import address
>>>
>>> address.validate_address('[email protected]')
foo@mailgun.com

To validate an address list:

>>> from flanker.addresslib import address
>>>
>>> address.validate_list(['[email protected], [email protected], @mailgun.com'], as_tuple=True)
([foo@mailgun.com, bar@mailgun.com], ['@mailgun.com'])

MIME Parsing

For the following examples, message_string will be set to the following MIME message:

MIME-Version: 1.0
Content-Type: multipart/alternative; boundary=001a11c1d71697c7f004e6856996
From: Bob <[email protected]>
To: Alice <[email protected]>
Subject: hello, world
Date: Mon, 16 Sep 2013 12:43:03 -0700

--001a11c1d71697c7f004e6856996
Content-Type: text/plain; charset=us-ascii

Hello, *Alice*

--001a11c1d71697c7f004e6856996
Content-Type: text/html; charset=us-ascii

<p>Hello, <b>Alice</b></p>

--001a11c1d71697c7f004e6856996--

To parse a MIME message:

>>> from flanker import mime
>>>
>>> msg = mime.from_string(message_string)

MIME message headers (unicode multi-value dictionary with headers):

>>> from flanker import mime
>>>
>>> msg = mime.from_string(message_string)
>>> msg.headers.items()
[('Mime-Version', '1.0'),
 ('Content-Type',
  ('multipart/alternative', {'boundary': u'001a11c1d71697c7f004e6856996'})),
 ('From', 'Bob <[email protected]>'),
 ('To', 'Alice <[email protected]>'),
 ('Subject', 'hello, world'),
 ('Date', 'Mon, 16 Sep 2013 12:43:03 -0700')]

Useful content_type member with predicates:

>>> from flanker import mime
>>> msg = mime.from_string(message_string)
>>>
>>> msg.content_type.is_multipart()
True
>>>
>>> msg.content_type.is_singlepart()
False
>>>
>>> msg.content_type.is_message_container()
False

Decoded body of a message:

>>> from flanker import mime
>>> msg = mime.from_string(message_string)
>>>
>>> # None because message is multipart
>>> print msg.body
None
>>>
>>> for part in msg.parts:
       print 'Content-Type: {} Body: {}'.format(part, part.body)

Content-Type: (text/plain) Body: Hello, *Alice*
Content-Type: (text/html) Body: <p>Hello, <b>Alice</b></p>

>>> # None because no enclosed messages exist
>>> print msg.enclosed
None
Comments
  • WIP: Make flanker compatible with python 3

    WIP: Make flanker compatible with python 3

    Work In Progress

    Make flanker compatible with python 3 (while maintaining python 2 compatibility)

    Depends on mailgun/expiringdict#15 and mailgun/dnsq#7.

    opened by salehe 14
  • Accept non-ascii attachment name

    Accept non-ascii attachment name

    Non-ascii param will no longer raise DecodingError, as shown in the old broken_ctype_test. This bug can be reproduced by sending an attachment in gmail with non-ascii name. The bug is fixed by tuning regex. Non-ascii params still have to be double quoted.

    opened by carsonip 10
  • Incrase max ops 10x

    Incrase max ops 10x

    In practice there are actually messages with over 500 attachments. We've seen this at Nylas while running our sync engine that provides an email API.

    You can simulate the failure with this script:

    from flanker import mime
    import uuid
    
    msg = mime.create.multipart('alternative')
    msg.append(
        mime.create.text('plain', 'Hello'),
        mime.create.text('html', '<html>Hello</html>'))
    
    text_msg = msg
    msg = mime.create.multipart('mixed')
    
    msg.append(text_msg)
    
    for a in range(5000):
        # Disposition should be inline if we add Content-ID
        msg.append(mime.create.attachment(
            'text/plain',
            str(uuid.uuid4())))
    
    
    msg.headers['Subject'] = 'Breaks Flanker'
    msg.headers['To'] = u'[email protected]'
    msg.headers['From'] = '[email protected]'
    
    rfcmsg = msg.to_string()
    
    break_msg = mime.from_string(rfcmsg)
    

    Throws

    Traceback (most recent call last):
      File "too_many_attachments.py", line 27, in <module>
        break_msg = mime.from_string(rfcmsg)
      File "/Users/mg/workspace/sync-engine/flanker/flanker/mime/create.py", line 82, in from_string
        return scanner.scan(string)
      File "/Users/mg/workspace/sync-engine/flanker/flanker/mime/message/scanner.py", line 24, in scan
        return traverse(Start(), TokensIterator(tokens, string))
      File "/Users/mg/workspace/sync-engine/flanker/flanker/mime/message/scanner.py", line 96, in traverse
        parts.append(traverse(token, iterator, content_type))
      File "/Users/mg/workspace/sync-engine/flanker/flanker/mime/message/scanner.py", line 34, in traverse
        iterator.check()
      File "/Users/mg/workspace/sync-engine/flanker/flanker/mime/message/scanner.py", line 270, in check
        self.opcount, _MAX_OPS))
    flanker.mime.message.errors.DecodingError: Too many parts: 501, max is 500
    

    Obviously one can create an email message with a nearly arbitrary number of attachments, but with our entire mail corpus, we haven't seen any that hit the limit in this commit. It's also worth noting that we have not experienced any "runaway" parsing that would require this limit check in the first place. (Perhaps it was only used when Flanker was being developed?)

    opened by grinich 10
  • parsetab module is not always generated property by ply

    parsetab module is not always generated property by ply

    It seems that there is some race condition with parsetab module that PLY generates. On flanker-0.7.2 we are getting the error that reproduces randomly in identical environments:

        from flanker.addresslib import address
    runtime/lib/python2.7/site-packages/flanker/addresslib/address.py:46: in <module>
        from flanker.addresslib.parser import (Mailbox, Url, mailbox_parser,
    runtime/lib/python2.7/site-packages/flanker/addresslib/parser.py:162: in <module>
        start='mailbox', errorlog=log)
    runtime/lib/python2.7/site-packages/ply/yacc.py:3287: in yacc
        read_signature = lr.read_table(tabmodule)
    runtime/lib/python2.7/site-packages/ply/yacc.py:1985: in read_table
        if parsetab._tabversion != __tabversion__:
    E   AttributeError: 'module' object has no attribute '_tabversion'
    

    parsetab.py file is correct and has all expected variables. However parsetab.pyc seems to be an empty module. After removing pyc file from flanker.addresslib import address works correctly.

    May be it is worth to add a generated parsetab.py into the package distribution?

    opened by rmihael 9
  • Ignore malformed multipart/alternative boundaries for multipart/report messages

    Ignore malformed multipart/alternative boundaries for multipart/report messages

    This fixes a special case where Google would return a multipart/report message indicating that a message delivery failed. The automated email response contained a message/rfc822 part (quoting the original message that failed to send) that has a multipart/alternative boundary with no boundary header, causing parsing to fail.

    Here is a sample python program that reproduces this bug. To be clear, the portion of this raw message that causes parsing to fail is the following line:

    Content-Type: multipart/alternative; boundary=f403045f50f42690580546f0cb4d

    since it isn't followed by any actual boundary.

    from flanker import mime
    
    # Raw mesage from Google when message delivery fails
    google_delivery_failed = """Delivered-To: [email protected]
    Received: by 10.194.110.130 with SMTP id ia2csp2041405wjb;
            Wed, 25 Jan 2017 12:08:18 -0800 (PST)
    X-Received: by 10.28.226.67 with SMTP id z64mr22592117wmg.137.1485374898494;
            Wed, 25 Jan 2017 12:08:18 -0800 (PST)
    Return-Path: <>
    Received: from mail-wm0-x245.google.com (mail-wm0-x245.google.com. [2a00:1450:400c:c09::245])
            by mx.google.com with ESMTPS id z104si28099490wrb.58.2017.01.25.12.08.18
            for <[email protected]>
            (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
            Wed, 25 Jan 2017 12:08:18 -0800 (PST)
    Received-SPF: pass (google.com: best guess record for domain of [email protected] designates 2a00:1450:400c:c09::245 as permitted sender) client-ip=2a00:1450:400c:c09::245;
    Authentication-Results: mx.google.com;
           dkim=pass [email protected];
           spf=pass (google.com: best guess record for domain of [email protected] designates 2a00:1450:400c:c09::245 as permitted sender) smtp.helo=mail-wm0-x245.google.com;
           dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com
    Received: by mail-wm0-x245.google.com with SMTP id d140so40596672wmd.4
            for <[email protected]>; Wed, 25 Jan 2017 12:08:18 -0800 (PST)
    DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
            d=googlemail.com; s=20161025;
            h=from:to:auto-submitted:subject:references:in-reply-to:message-id
             :date;
            bh=UZlFrv9E6e9cIhpOuMLk3yb0RpRweat0tRiPv8ekqz0=;
            b=eWX08lml0oddE/oHJmpNqfTcF60QWnyZMKpOsx3LiQ8AAl20POAt1CO18pvDuUCiEM
             lqT1rb3fXbiC+/aXw61QGh2Eb7f003RTybbxOEG5mwNzzCU5yNghXRkMLBQx3AOZDBIm
             1ZBvPgVSR65J86lU+USxMr+Ol61LPnOROpTZFckLAfs991A6oTfjoddqGeNOHe5/GM4o
             8OLlJOhKoeTY13Os17mKmUEoZXPI8YgB+/b6OHmN2cwLBRIowVFd1sURsBL7+S4JMHhm
             zxt0rXG5xvA4U00yvxni/Zr51TWTWJ7AAS+ei80hZwOq5nuriYVjLkHXhA7ZZR249VdZ
             Mmfg==
    X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
            d=1e100.net; s=20161025;
            h=x-gm-message-state:from:to:auto-submitted:subject:references
             :in-reply-to:message-id:date;
            bh=UZlFrv9E6e9cIhpOuMLk3yb0RpRweat0tRiPv8ekqz0=;
            b=PaW6CKnvbWr1aBixbBDFVbJcf2EKgBxdm8RE6RFD47DwwjkoZKVv5xOqZhPNyVdmyN
             67IGafwEcshPy+8vSt1zchGA7TJGs5vTWK89WadpI9eGyAPPIwNZ4Xd2/vB2uvK3PIXG
             Nujf0pRfR4gPP9kEvxq3NB1HXzKn0SujgEPl8uWVaav15J23M0Txjmm7y/8dEiJz4ray
             yrzNyVXf65830O89FKNtlHTEzAtXISHuAt6+hApvfZXgEXUtXSadoF2pNg1RhjyzOBK9
             L3doUPYO4udZvg6WCsKKeWcPnkPKeCmcjIpqJimRY8a85pWj08GpNeMxUGhAHwUVhTns
             TJdg==
    X-Gm-Message-State: AIkVDXKdWCy00iCWlxjYLB1rrDw/224nOWKc8uQoeejDhu1/qVoAoil61X2Q32xmquhJpHk/OIeAy8+09x6q6b4d74xWzMTue470/X0=
    X-Received: by 10.223.150.183 with SMTP id u52mr34057945wrb.180.1485374898376;
            Wed, 25 Jan 2017 12:08:18 -0800 (PST)
    Content-Type: multipart/report; boundary=f403045f50f42d03f10546f0cb14; report-type=delivery-status
    Return-Path: <>
    Received: by 10.223.150.183 with SMTP id u52mr37079428wrb.180; Wed, 25 Jan
     2017 12:08:18 -0800 (PST)
    From: Mail Delivery Subsystem <[email protected]>
    To: [email protected]
    Auto-Submitted: auto-replied
    Subject: Delivery Status Notification (Failure)
    References: <CAEY6-q_Ev34Fhib_jSr=8M93qwoDqxkt+CXj2M1XNMDm64xyTg@mail.gmail.com>
    In-Reply-To: <CAEY6-q_Ev34Fhib_jSr=8M93qwoDqxkt+CXj2M1XNMDm64xyTg@mail.gmail.com>
    Message-ID: <[email protected]>
    Date: Wed, 25 Jan 2017 12:08:18 -0800 (PST)
    
    --f403045f50f42d03f10546f0cb14
    Content-Type: multipart/related; boundary=f403045f50f42d0bd40546f0cb1a
    
    --f403045f50f42d0bd40546f0cb1a
    Content-Type: multipart/alternative; boundary=f403045f50f42d0bdc0546f0cb1b
    
    --f403045f50f42d0bdc0546f0cb1b
    Content-Type: text/plain; charset=UTF-8
    
    
    ** Address not found **
    
    Your message wasn't delivered because the domain faketestemail.xxx couldn't be found. Check for typos or unnecessary spaces and try again.
    
    
    
    The response from the remote server was:
    DNS Error: 42534961 DNS type 'mx' lookup of faketestemail.xxx responded with code NXDOMAIN
    Domain name not found: faketestemail.xxx
    
    --f403045f50f42d0bdc0546f0cb1b
    Content-Type: text/html; charset=UTF-8
    
    
    <html>
    <head>
    <style>
    * {
    font-family:Roboto, "Helvetica Neue", Helvetica, Arial, sans-serif;
    }
    </style>
    </head>
    <body>
    <table cellpadding="0" cellspacing="0" class="email-wrapper" style="padding-top:32px;background-color:#ffffff;"><tbody>
    <tr><td>
    <table cellpadding=0 cellspacing=0><tbody>
    <tr><td style="max-width:560px;padding:24px 24px 32px;background-color:#fafafa;border:1px solid #e0e0e0;border-radius:2px">
    <img style="padding:0 24px 16px 0;float:left" width=72 height=72 alt="Error Icon" src="cid:icon.png">
    <table style="min-width:272px;padding-top:8px"><tbody>
    <tr><td><h2 style="font-size:20px;color:#212121;font-weight:bold;margin:0">
    Address not found
    </h2></td></tr>
    <tr><td style="padding-top:20px;color:#757575;font-size:16px;font-weight:normal;text-align:left">
    Your message wasn't delivered because the domain faketestemail.xxx couldn't be found. Check for typos or unnecessary spaces and try again.
    </td></tr>
    </tbody></table>
    </td></tr>
    </tbody></table>
    </td></tr>
    <tr style="border:none;background-color:#fff;font-size:12.8px;width:90%">
    <td align="left" style="padding:48px 10px">
    The response from the remote server was:<br/>
    <p style="font-family:monospace">
    DNS Error: 42534961 DNS type &#39;mx&#39; lookup of faketestemail.xxx responded with code NXDOMAIN
    Domain name not found: faketestemail.xxx
    </p>
    </td>
    </tr>
    </tbody></table>
    </body>
    </html>
    
    --f403045f50f42d0bdc0546f0cb1b--
    --f403045f50f42d0bd40546f0cb1a
    Content-Type: image/png; name="icon.png"
    Content-Disposition: attachment; filename="icon.png"
    Content-Transfer-Encoding: base64
    Content-ID: <icon.png>
    
    iVBORw0KGgoAAAANSUhEUgAAAJAAAACQCAYAAADnRuK4AAAACXBIWXMAABYlAAAWJQFJUiTwAAAA
    GXRFWHRTb2Z0d2FyZQBBZG9iZSBJbWFnZVJlYWR5ccllPAAABTdJREFUeNrsnD9sFEcUh5+PRMqZ
    yA0SPhAUQAQFUkyTgiBASARo6QApqVIkfdxGFJFSgGhJAUIiBaQB0ZIOKVCkwUgURjIg2fxL4kS+
    YDvkbC/388bi8N16Z4/d7J/5PsniuD3fyePP772ZeTsDQRAYQL/UGAJAIEAgQCBAIAAEAgQCBAIE
    AkAgyJT3Mv+Eq7vYK8mTE+MDRCAghQECAeRQA5V2ZOpmg5vDx3NPzRbmGRMEcmTrEbNNB8zWfRD+
    f/Efs2e3zCZvMjaksBg27TfbcuSNPEKP9ZyuAQKtHX2O9ncNgWC57umMPKvRNb0GEKgnLoUyxTQC
    rcns0/6uIRAs8/hGf9cQCJZpTpjdO2f25/03z+mxntM1eLtsZAgiUtX4JcaBCAQIBAgECARQ8CJa
    G5jab4J4pm4WZmO3OALVh802fIwcLkyPkcKAGggAgQCBAIEAgQCBABAIEAjKA/1AnahhbO5FdOOY
    VsrrDbPBYcYKgf5D2wLaV3p+22xh1u17tO3S+DTcvxvagUDeivPgx/a/95J/73w7Sj26Hn4pKo2M
    ehuV/KyBJM6d0f7k6RKx/R63vvL2tmf/ItDdM2ZTP6f7nkp9Y2fDx1v9akmpIU+KSCLVUghUQfSL
    zVKeTklbLxGoctw/nzC5rw8L5KRNbkpnKq6pgSqEClzNnFzY+XnYWrt6VpVk1vbwWvg+RKCKMOUw
    Q1LEOXA+/MX3mpJvGDHb265xtnzmFoUK1HaKQGlMtePYM+q2KKjXuaS1NJYIEKgI8jhEgqHt4cqy
    Ky53j3hyHz2bqSLp2o2LbJ7MxKovkGqXteoWpaOk96O9/yF/dF7NwlS36AuIQIBA5celQK4PIxBE
    4LLzrtoLgaALdSy6CJRkWQCBPGLsTHznomZ9nszUECgJ2ml3WWHe+QVFNPSQx6UdZNtxr9pbEShN
    eTTz8mQXHoHSlke7+Z+c9m6VGoHSkEfs/trLW3wQKApN1V3lGfnGu2Z6BFoLtYCs3GWBPAiUCLVh
    /HoaeRCoT9R873KLM/IgUBfapnCpe5AHgXry4pf412ihEHkQqCdxd5VqrcezhUIESsJMTJ+Pdthp
    Z0WgyNlXXPHc2Mc4IVAELl2Gnh8mhUDvCkfbIVAkcbf/aOoO3fMKhqAD3frTa4quwpn0hUDOkQhI
    YYBAgECAQAAU0QlYObl+5Ug8NcprZkZxjUCxRPVA6zmtEXHCBykskrhjgHXN09PoEcgFl4M4H11j
    nBAoApcj6ZoPGScEAgTKApcDoTw5sgWB+sGlz1n90IBAPdE6j1o21PfcC11jLagL1oFWRyGlKU3p
    OxcSJQ7NZAjkhHp/uG2HFAYIBAgECASAQIBAgECAQAAIBOkxEARBtp9wdVfAMOfIifEBIhCQwgCB
    ABAI0oV2jhxZ+nfBatuPZfgBCy0Eqqo8c01b+uu51XZvzOgDWoHNTGR+pCwpLEd5svuAZXlO2uEr
    PyEQ8hRWHgRCHmqg0sjTnLalv6crJQ8C/U8stqNO0I4+VZOHFIY8COS1PGL2ybd5yUMKK7s8zYmL
    dujyd3n+nESgcsvzZd4/KwIhDwIhT35QA6UyE1qyxZnfvJMHgdKS549JC1qvvJOHFIY8CFR5eV5O
    XimqPAhUdHnmfx+zgxdOFXkoqIGKKs/cswnb/8Oeog8HEai48nxUhiFBIORBIOShBioskkbySCLk
    IQIhDwIhj28p7FApR6b1qlEbHGpkO/rr6215vi/zH1r2x7tApSGFAQIBAgECAQIBIBAgECAQIBBA
    LK8FGADCTxYrr+EVJgAAAABJRU5ErkJggg==
    --f403045f50f42d0bd40546f0cb1a--
    --f403045f50f42d03f10546f0cb14
    Content-Type: message/delivery-status
    
    Reporting-MTA: dns; googlemail.com
    Arrival-Date: Wed, 25 Jan 2017 12:08:17 -0800 (PST)
    X-Original-Message-ID: <CAEY6-q_Ev34Fhib_jSr=8M93qwoDqxkt+CXj2M1XNMDm64xyTg@mail.gmail.com>
    
    Final-Recipient: rfc822; [email protected]
    Action: failed
    Status: 4.0.0
    Diagnostic-Code: smtp; DNS Error: 42534961 DNS type 'mx' lookup of faketestemail.xxx responded with code NXDOMAIN
     Domain name not found: faketestemail.xxx
    Last-Attempt-Date: Wed, 25 Jan 2017 12:08:18 -0800 (PST)
    
    --f403045f50f42d03f10546f0cb14
    Content-Type: message/rfc822
    
    DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
            d=gmail.com; s=20161025;
            h=mime-version:from:date:message-id:subject:to;
            bh=sN7wtuHkt2/TNQC15owDWy6WHFRWvOL9FfumQpG2Hoo=;
            b=VTvZhlfywjiKb69vgWMpLXYmxlw5IKk0nOkSdueVL4BTkpf8f7tArFHMatItTYnpIV
             0u92ysWbjsUoNKT++P3Twmgd3xpFL4UBJKJxq6acRTUKqNCur9jdKpgLGnywdRL95f5a
             4zYTbmPf1V6u6iDP7ijb6KP7lAy7w7UF1aXKFr+1XASr1Jb+XeSiQIh05zxlN4/wXlwS
             0ZkYOKZn2e71BHyAUWuNX/lDa5E72wpNnq6LbivuB2SQMLJgb+z41SrAI00ZfDQgwWhs
             s6GRPEZxZ+gKXvDqpj7Z2JDrhDi0laU7MLyWN7YLPIKXw7JQijgXfd27ZWISYCe7EE4O
             /JBw==
    X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
            d=1e100.net; s=20161025;
            h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
            bh=sN7wtuHkt2/TNQC15owDWy6WHFRWvOL9FfumQpG2Hoo=;
            b=EQqihY9ManAuK5Vnb8QeiphhvpMA0206U+9VtZsaAiYj7EcQr+lbKm2C44H+FC9iYc
             kRdldQB3YwZUcJF51wXk6ANsQMFKqGrlHZSet/loERsDYn+rEQSexS3nuYVwanaTueWy
             4lfM1LCOR4aGN2WTDD8BVzd/VGiDZUWYbXal8Msa2p0Slt+80qAk2tRv6yV7YB6XToyD
             trGJIjfdeJVXxt/uPEUY46NHEFS2T1aqFqxdQbAeCj1lYguEsAyxf9GpTlMfqrlTgLse
             cqonOBtyqFhwk6+uKdqQ8uI4OoVtgsgaFwObcJgcWjJqggS68pQYhMl1ySqxLHhgu6lW
             NxHg==
    X-Gm-Message-State: AIkVDXKldzXzUmU2Tkd9mjfh+Je/hsCIa2uA3SpSCuyE+QZFwNohM4rWEGIzuOESP7Afsa4UfwhtMLpW4D16Rw==
    X-Received: by 10.223.150.183 with SMTP id u52mr34057921wrb.180.1485374897948;
     Wed, 25 Jan 2017 12:08:17 -0800 (PST)
    MIME-Version: 1.0
    From: Michael <[email protected]>
    Date: Wed, 25 Jan 2017 20:08:07 +0000
    Message-ID: <CAEY6-q_Ev34Fhib_jSr=8M93qwoDqxkt+CXj2M1XNMDm64xyTg@mail.gmail.com>
    Subject: fake send
    To: [email protected]
    Content-Type: multipart/alternative; boundary=f403045f50f42690580546f0cb4d
    
    fake send
    
    --f403045f50f42d03f10546f0cb14--
    """
    
    parsed_message = None
    
    try:
        parsed_message = mime.from_string(google_delivery_failed)
    except mime.DecodingError() as e:
        print e
    
    print parsed_message.to_string()
    
    opened by pfista 9
  • If unable to contact a host, don't cache that result.

    If unable to contact a host, don't cache that result.

    Purpose

    Right now, if we fail to connect to a MX server, we cache that as unable to connect. This can cause issues if the inability to connect was either a DNS timeout or some other temporary network issues. This commit changes this behavior to not cache the result of a MX lookup.

    Implementation

    In the function mail_exchanger_lookup in validate.py remove the lines that cache lookup values that were False.

    opened by glycerine 8
  • Fix for decoding errors if the list includes a non-ASCII string.

    Fix for decoding errors if the list includes a non-ASCII string.

    When decoding, addresslib shouldn't assume strings are only ASCII.

    >>> from flanker.addresslib import address
    >>> address.parse_list([u'A <\[email protected]>'])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Library/Python/2.7/site-packages/flanker/utils.py", line 99, in wrapper
        return_value = f(*args, **kwargs)
      File "/Library/Python/2.7/site-packages/flanker/addresslib/address.py", line 133, in parse_list
        address_list = ', '.join([str(addr) for addr in address_list])
    UnicodeEncodeError: 'ascii' codec can't encode character u'\xfc' in position 3: ordinal not in range(128)
    

    Discovered via https://github.com/inboxapp/inbox/issues/102

    opened by grinich 8
  • Add support for Unicode strings in email addresses

    Add support for Unicode strings in email addresses

    This PR adds support for non-ASCII characters in the local and domain parts of email addresses and domain literals for the domain part. It completely rewrites the lexer and parser based on Ply and heavily refactors of the address module. The previous lexer module, tokenizer.py, is kept because some of the domain specific validations use it. Those should be the subject of a later PR.

    As far as I'm aware, the public interfaces in the address module should be unchanged and all the addresses that validated previously should still validate properly.

    The only potentially breaking change is that I've removed the relaxed vs. strict parsing of mailing lists. The trade-off for having a more flexible parser is that we can't make as many assumptions about an unparsable list. parse_list() can still return a tuple of parsed and unparsed addresses but only if the input is a list of discrete addresses. It's up to services that use this module to either format user input into a list or return an appropriate error to the user so that they can reformat and resubmit the list.

    I've also added a debug terminal to parser.py that can be invoked with python flanker/addresslib/parser.py. Given an address it will show the list of tokens, parsing behaviour, and final result. Sample output:

    flanker> [email protected]
    
    Tokens list:
    
    LexToken(ATEXT,'foo',1,0)
    LexToken(AT,'@',1,3)
    LexToken(ATEXT,'bar',1,4)
    LexToken(DOT,'.',1,7)
    LexToken(ATEXT,'com',1,8)
    
    Parsing behavior:
    
    INFO:__main__:PLY: PARSE DEBUG START
    INFO:__main__:Action : Reduce rule [ofwsp -> <empty>] with [] and goto state 3
    INFO:__main__:Result : <str @ 0x7f89b88e6508> ('')
    INFO:__main__:Action : Reduce rule [ofwsp -> <empty>] with [] and goto state 42
    INFO:__main__:Result : <str @ 0x7f89b88e6508> ('')
    INFO:__main__:Action : Reduce rule [atom -> ofwsp ATEXT ofwsp] with ['','foo',''] and goto state 11
    INFO:__main__:Result : <str @ 0x7f89b002aaa8> ('foo')
    INFO:__main__:Action : Reduce rule [local_part -> atom] with ['foo'] and goto state 6
    INFO:__main__:Result : <str @ 0x7f89b002aaa8> ('foo')
    INFO:__main__:Action : Reduce rule [ofwsp -> <empty>] with [] and goto state 44
    INFO:__main__:Result : <str @ 0x7f89b88e6508> ('')
    INFO:__main__:Action : Reduce rule [dot_atom_text -> ATEXT DOT ATEXT] with ['bar','.','com'] and goto state 19
    INFO:__main__:Result : <str @ 0x7f89afb70f00> ('bar.com')
    INFO:__main__:Action : Reduce rule [ofwsp -> <empty>] with [] and goto state 39
    INFO:__main__:Result : <str @ 0x7f89b88e6508> ('')
    INFO:__main__:Action : Reduce rule [dot_atom -> ofwsp dot_atom_text ofwsp] with ['','bar.com',''] and goto state 45
    INFO:__main__:Result : <str @ 0x7f89afb70f00> ('bar.com')
    INFO:__main__:Action : Reduce rule [domain -> dot_atom] with ['bar.com'] and goto state 43
    INFO:__main__:Result : <str @ 0x7f89afb70f00> ('bar.com')
    INFO:__main__:Action : Reduce rule [addr_spec -> local_part AT domain] with ['foo','@','bar.com'] and goto state 7
    INFO:__main__:Result : <Mailbox @ 0x7f89afb6af70> (Mailbox(display_name='', local_part='foo ...)
    INFO:__main__:Action : Reduce rule [mailbox -> addr_spec] with [<Mailbox @ 0x7f89afb6af70>] and goto state 9
    INFO:__main__:Result : <Mailbox @ 0x7f89afb6af70> (Mailbox(display_name='', local_part='foo ...)
    INFO:__main__:Action : Reduce rule [mailbox_or_url -> mailbox] with [<Mailbox @ 0x7f89afb6af70>] and goto state 8
    INFO:__main__:Result : <Mailbox @ 0x7f89afb6af70> (Mailbox(display_name='', local_part='foo ...)
    INFO:__main__:Action : Reduce rule [mailbox_or_url_list -> mailbox_or_url] with [<Mailbox @ 0x7f89afb6af70>] and goto state 13
    INFO:__main__:Result : <list @ 0x7f89afb830e0> ([Mailbox(display_name='', local_part='fo ...)
    INFO:__main__:Done   : Returning <list @ 0x7f89afb830e0> ([Mailbox(display_name='', local_part='fo ...)
    INFO:__main__:PLY: PARSE DEBUG END
    
    Result:
    
    [Mailbox(display_name='', local_part='foo', domain='bar.com')]
    
    opened by b0d0nne11 7
  • Improved py36 support

    Improved py36 support

    The changes made here are dealing with the differences between iteration functions in py3 and py2. Specifically, dnsq returns a result of a filter function from its mx_hosts_for function. In Python 2 it's a list, but for py3 it's an iterator. Later flanker tries to get a len of that list which is impossible with an iterator. So this pull request deals with this by converting iterator to a list.

    Other than that there are some improvements in tests.

    opened by eserge 6
  • regex version pin breaks Python 3 support

    regex version pin breaks Python 3 support

    Currently regex has a hard version pin on a very old version of the regex package. This version does not support Python 3, which results in flanker also not being usable in Python 3 projects.

    The version pin has a comment that indicates that this is done for performance reasons. I am wondering a few things:

    • Is there a benchmark script that one can run to test this performance problem? That would make it possible to discuss this with the regex developers.
    • Which regex version showed this degradation?
    • For many sites slow performance for flanker is not problematic. Can you consider doing the same pin/unpinned trick you added for #20 for regex as well?
    opened by wichert 6
  • ImportError: No module named paste.util.multidict

    ImportError: No module named paste.util.multidict

    FYI. :)

    โ†’ sudo pip install flanker
    [โ€ฆ]
    
    
    
        442 warnings generated.
        cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. build/temp.macosx-10.9-intel-2.7/Python2/_regex.o -o build/lib.macosx-10.9-intel-2.7/_regex.so
    
      Running setup.py install for dnspython
    
    Successfully installed flanker chardet dnsq expiringdict mock nose Paste redis regex dnspython
    Cleaning up...
    

    Then on the CLI

    โ†’ python
    Python 2.7.5 (default, Aug 25 2013, 00:04:04) 
    [GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> from flanker.addresslib import address
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/Library/Python/2.7/site-packages/flanker/addresslib/address.py", line 38, in <module>
        import flanker.addresslib.parser
      File "/Library/Python/2.7/site-packages/flanker/addresslib/parser.py", line 79, in <module>
        from flanker.mime.message.headers.encoding import encode_string
      File "/Library/Python/2.7/site-packages/flanker/mime/__init__.py", line 61, in <module>
        from flanker.mime.message.errors import DecodingError, EncodingError, MimeError
      File "/Library/Python/2.7/site-packages/flanker/mime/message/__init__.py", line 1, in <module>
        from flanker.mime.message.scanner import ContentType
      File "/Library/Python/2.7/site-packages/flanker/mime/message/scanner.py", line 4, in <module>
        from flanker.mime.message.headers import parsing, is_empty, ContentType
      File "/Library/Python/2.7/site-packages/flanker/mime/message/headers/__init__.py", line 1, in <module>
        from flanker.mime.message.headers.headers import MimeHeaders
      File "/Library/Python/2.7/site-packages/flanker/mime/message/headers/headers.py", line 1, in <module>
        from paste.util.multidict import MultiDict
    ImportError: No module named paste.util.multidict
    
    opened by karlcow 6
  • Add support for message/rfc822-headers content type

    Add support for message/rfc822-headers content type

    There is a bug in how flanker parses messages with the content type of message/rfc822-headers. This PR fixes this by treating the message/rfc822-headers type as a header container, just like text/rfc822-headers

    opened by tiger-zz 1
  • docs: Fix a few typos

    docs: Fix a few typos

    There are small typos in:

    • docs/API Reference.md
    • docs/User Manual.md
    • tests/addresslib/plugins/aol_test.py
    • tests/addresslib/plugins/gmail_test.py
    • tests/addresslib/plugins/google_test.py
    • tests/addresslib/plugins/hotmail_test.py
    • tests/addresslib/plugins/icloud_test.py
    • tests/addresslib/plugins/yahoo_test.py
    • tests/mime/message/headers/headers_test.py

    Fixes:

    • Should read occurring rather than occuring.
    • Should read referred rather than refered.
    • Should read operating rather than opearting.
    • Should read beginning rather than begining.

    Semi-automated pull request generated by https://github.com/timgates42/meticulous/blob/master/docs/NOTE.md

    opened by timgates42 1
  • Mismatching version and tag versions

    Mismatching version and tag versions

    It seems that the version stayed on 0.9.11 in https://github.com/mailgun/flanker/blob/master/setup.py#L13 While the tag 0.9.14 was released https://github.com/mailgun/flanker/tags Not a big deal but that's a bit confusing.

    opened by guillaume-miara 0
  • Suppress unreachable symbol warnings; don't write cached tables

    Suppress unreachable symbol warnings; don't write cached tables

    Address two outstanding issues that are especially a problem when using flanker in a frozen application (i.e. using pyinstaller).

    #231 Disable warnings on first run

    When addresslib is first imported, the yacc parser gets run and logs a bunch of warnings about missing symbols. This pollutes the stdout (or maybe stderr) of the application using it. And when freezing with pyInstaller, the directory with cached tables is removed between runs, so this happens on every run. I have no idea if these warnings are themselves a bug, but they shouldn't be exposed like this and setting check_recursion=False suppresses them.

    #208 Don't generate cached parser tables

    When running in a read-only filesystem, yacc will fail to write out the cached parser tables. This affects, for example, AWS Lambda and pyInstaller builds. It's not fatal, but it does generate an error in the application output each time.

    There is a cost to write_tables=False, since we will have to re-parse on the next startup. But it's worth noting that the next version of the yacc module will do away with caching entirely, and the author considers it of negligible performance benefit:

    PLY no longer writes cached table files. Honestly, the use of the cached files made more sense when I was developing PLY on my 200Mhz PC in 2001. It's not as much as an issue now. For small to medium sized grammars, PLY should be almost instantaneous. If you're working with a large grammar, you can arrange to pickle the associated grammar instance yourself if need be.

    Testing

    Have verified no regressions in nosetests.

    opened by mikebveil 1
Releases(v0.9.14)
Generate Email, Register for anything, Get the OTP/Link

OTE : One Time Email Introduction ote is a command line utility that generates temporary email address and automatically extracts OTPs or confirmation

Somdev Sangwan 457 Jan 03, 2023
Fast Anonymous Email Sending Tool

Email-Fake Fast Anonymous Email Sending Tool ๐Ÿ† Github Statistics : Termux For Install: pkg install python pkg install python2 git clone https://githu

Aryan 7 May 28, 2022
Email-osint - Email OSINT tool written in python3

Email-osint - Email OSINT tool written in python3

Surtains 7 Nov 28, 2022
An OSINT program that allows you to uncover a censored domain in an email adress

An OSINT program that allows you to uncover a censored domain in an email adress. Useful when you extract email from Instagram or Twitter password recovery function.

aet 3 Dec 16, 2021
Email-bomber - Email bomber unlike other email bombers u don't need your gmail email id to use this

Email-bomber - Email bomber unlike other email bombers u don't need your gmail email id to use this

rfeferfefe 82 Dec 17, 2022
A simple free API that allows you to extract abuse emails from IPs.

Abuse-Email-API A simple free API that allows you to extract abuse emails from IPs. also isnt worth 500 dollars :) Requirements A Debian based OS The

Keratin 1 Dec 20, 2021
Send email in Python conveniently for gmail using yagmail

yagmail -- Yet Another GMAIL/SMTP client For the asynchronous asyncio version, look here: https://github.com/kootenpv/aioyagmail The goal here is to m

Pascal van Kooten 2.4k Dec 31, 2022
Spam-bot - Simple email-spammer discord bot

๐Ÿ“ Functional [ โœ”๏ธ ] Premium system via .json [ โœ”๏ธ ] Spammer [ โœ”๏ธ ] Validater [ โœ”๏ธ ] Discord bot โ“ How to launch โžก๏ธ 1) Make discord bot โžก๏ธ 2) Paste to

1 Feb 18, 2022
A simple email sender

Email-Sender Un semplice Email-Sender che utilizza il modulo smtplib con aggiunta di interfaccia grafica realizzata con il modulo tkinter Per il corre

Vincenzo Caliendo 0 Jan 14, 2022
ParaskinioTouristOffices - This program sends a message to various email adresses

ParaskinioTouristOffices This program sends a message to various email adresses.

Odysseas Psomaderis 2 Feb 11, 2022
This python script will generate passwords for your emails, With certain lengths, And saves them into plain text files.

How to use. Change the Default length of genereated password in default.length.txt Type the email for your account. Type the website that the email an

2 Dec 26, 2021
A Django email backend for Amazon's Simple Email Service

Django-SES Info: A Django email backend for Amazon's Simple Email Service Author: Harry Marr (http://github.com/hmarr, http://twitter.com/harrymarr) C

882 Dec 29, 2022
automatic mails sender with attachments

ุฃุฒุนุฌู†ูŠ ู„ูŠู† ุชุฏุฑุจู†ูŠ Automatic mails sender with attachments. Note: You need to have gmail account & and you need to turn on "Less secure app access" set

6 Dec 30, 2022
Python script for imap, pop3, smtp and IPv4 analyze

Python script for imap, pop3, smtp and IPv4 analyze

Vladislav Kotletkin 1 Jan 30, 2022
Python email address and Mime parsing library

Flanker - email address and MIME parsing for Python Flanker is an open source parsing library written in Python by the Mailgun Team. Flanker currently

Mailgun Team 1.6k Dec 29, 2022
Disposable Temporary Email (Python Library)

Disposable Temporary Email (Python Library)

krypton 13 Nov 24, 2022
EmailAll - a powerful Email Collect tool

EmailAll A powerful Email Collect tool 0x1 ไป‹็ป ๐Ÿ˜ฒ EmailAll is a powerful Email Co

473 Dec 22, 2022
Bulk Email and certificate sending application

demir.ai E-mail services This application allows you to send automatic mass mail and automatic mass certificates to the people in your mailing list, m

Ahmet Furkan DEMIR 16 Nov 01, 2022
Email pass separator

email-pass-separator hii check out our new tool in kali linux use 'filename ' Dont forget to put inverted comma email:password separator Image Command

Hackers Tech 2 Sep 22, 2021
send email & telegram message whenever an analog in is recieved

send email & telegram message whenever an analog in is recieved (so when attached to an alarm siren out it will alert via mail)

Naor Livne 2 Feb 11, 2022