A tool to automatically fix Django deprecations.

Is your feature request related to a problem? Please describe. I use django-tailwind for CSS in a project. One of the JS libraries it imports has some Python 2 code that it puts inside my theme/static_src/node_modules/ directory, which is already in my .gitignore. Because my project is using Python 3.8, the code modder throws syntax errors for those 23 files (which is technically correct).

Describe the solution you'd like I'd like the ability to exclude paths (via a command-line switch or other means) or have the option to have django-codemod skip files ignored by git.

Additional context N/A, but this tool is excellent!

Adding a feature to handle adding the on_delete argument to ForeignKey and OneToOneField types: https://docs.djangoproject.com/en/dev/releases/1.9/#foreignkey-and-onetoonefield-on-delete-argument

TODO:

• [x] Add some more tests
• [x] Address adding the models import if it doesn't exist
• [x] Support detecting the on_delete call when passed as the second position argument
• [x] Fix on_delete matcher which doesn't seem to be finding things right
• [x] Test internally with a complicated codebase 💃

Stretch:

• [ ] ~Add an input so folks can add in subclasses~ (Edit: Probably going to do this in a future PR)

Refs #23

I'm not sure if this is a bit controversial since the original explicitly only allowed either option, but I don't see harm in a "just bring me up to date fam" option (speaking of which, --all could be added separately).

• makes --removed-in and --deprecated-in repeatable
• allows mixing and matching the two * adds an --all option

I was building my first fixer, found at its current horrid state at https://github.com/browniebroke/django-codemod/pull/123 (4803651c26), then I ran the following on a file which definitely needs the update

djcodemod --removed-in 2.0 redirect_urls/utils.py

The file in question can be found at https://github.com/pmac/django-redirect-urls/

I go get a coffee, and get otherwise distracted, etc, etc. A few hours later I come back and find my machine is out of memory, and codemod is having fun in my pnpm global node_modules , which is rather large.

I am sure I will find out how and why, but it isnt immediately clear to me, and I dare say something needs to prevent this happening to other people.

I had thousands of these:

Codemodding /home/jayvdb/.asdf/installs/nodejs/8.14.0/.npm/pnpm-global/1/node_modules/.registry.npmjs.org/node-gyp/3.8.0/node_modules/node-gyp/gyp/pylib/gyp/xcode_ninja.py
Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/libcst/_parser/base_parser.py", line 152, in _add_token
    plan = stack[-1].dfa.transitions[transition]
KeyError: ReservedString(,)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/libcst/codemod/_cli.py", line 283, in _parallel_exec_process_stub
    input_tree = parse_module(
  File "/usr/lib/python3.8/site-packages/libcst/_parser/entrypoints.py", line 71, in parse_module
    result = _parse(
  File "/usr/lib/python3.8/site-packages/libcst/_parser/entrypoints.py", line 51, in _parse
    result = parser.parse()
  File "/usr/lib/python3.8/site-packages/libcst/_parser/base_parser.py", line 111, in parse
    self._add_token(token)
  File "/usr/lib/python3.8/site-packages/libcst/_parser/base_parser.py", line 187, in _add_token
    raise ParserSyntaxError(
libcst._exceptions.ParserSyntaxError: Syntax Error @ 31:17.
Incomplete input. Encountered ',', but expected ':'.

  except OSError, e:
                ^

Failed to codemod /home/jayvdb/.asdf/installs/nodejs/8.14.0/.npm/pnpm-global/1/node_modules/.registry.npmjs.org/node-gyp/3.8.0/node_modules/node-gyp/gyp/pylib/gyp/xcode_ninja.py

Here is the output at the end when it was killed (kinda nice that my oom killer did kill this and not my desktop apps, which it often targets first)

38m 42s 0.0540% complete, 1194h 25m 20s estimated for 1362939 files to go...Terminated

Describe the bug A clear and concise description of what the bug is. ImportError on an Apple M1 Mac.

libcst seems to be installed for x86_64, not the required arm64e.

To Reproduce Steps to reproduce the behavior:

% pipx --version

1.0.0

% pipx install django-codemod

  installed package django-codemod 1.10.0, Python 3.9.12
  These apps are now globally available
    - djcodemod
done! ✨ 🌟 ✨

% djcodemod run --deprecated-in 1.3 .

Running codemods: ActionCheckboxNameTransformer
Executing codemod...
Traceback (most recent call last):
  File "/Users/cclauss/.local/bin/djcodemod", line 8, in <module>
    sys.exit(djcodemod())
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/rich_click/rich_group.py", line 21, in main
    return super().main(*args, standalone_mode=False, **kwargs)
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/django_codemod/cli.py", line 188, in run
    call_command(command_instance, files)
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/django_codemod/cli.py", line 195, in call_command
    result = parallel_exec_transform_with_prettyprint(
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/libcst/codemod/_cli.py", line 623, in parallel_exec_transform_with_prettyprint
    parse_module(
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/libcst/_parser/entrypoints.py", line 109, in parse_module
    result = _parse(
  File "/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/libcst/_parser/entrypoints.py", line 42, in _parse
    from libcst.native import parse_expression, parse_module, parse_statement
ImportError: dlopen(/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/libcst/native.cpython-39-darwin.so, 0x0002):
    tried: '/Users/cclauss/.local/pipx/venvs/django-codemod/lib/python3.9/site-packages/libcst/native.cpython-39-darwin.so'
    (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e'))

Additional context Add any other context about the problem here.

Describe the bug

I'm migrating a medium-sized codebase, and I found this strange behavior: if the code has a "from django.utils import timezone" import, a ParserSyntaxError is raised

$ pre-commit run -a
djcodemod................................................................Failed
- hook id: djcodemod
- exit code: 1

Running codemods: AssignmentTagTransformer, CookieDateTransformer, FixedOffsetTransformer, FloatRangeFormFieldTransformer, FloatRangeModelFieldTransformer, InlineHasAddPermissionsTransformer, ModelsPermalinkTransformer, OnDeleteTransformer, QuerySetPaginatorTransformer, SignalDisconnectWeakTransformer, URLResolversTransformer
Executing codemod...
Finished codemodding 62 files!
 - Transformed 62 files successfully.
 - Skipped 0 files.
 - Failed to codemod 0 files.
 - 0 warnings were generated.
Running codemods: AssignmentTagTransformer, CookieDateTransformer, FixedOffsetTransformer, FloatRangeFormFieldTransformer, FloatRangeModelFieldTransformer, InlineHasAddPermissionsTransformer, ModelsPermalinkTransformer, OnDeleteTransformer, QuerySetPaginatorTransformer, SignalDisconnectWeakTransformer, URLResolversTransformer
Executing codemod...
Codemodding /home/luzfcb/projects/big_project/foobar/tests/test_managers.py
Traceback (most recent call last):
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/_parser/base_parser.py", line 152, in _add_token
    plan = stack[-1].dfa.transitions[transition]
KeyError: ReservedString(import)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/codemod/_cli.py", line 295, in _execute_transform
    output_tree = transformer.transform_module(input_tree)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/codemod/_command.py", line 88, in transform_module
    tree = self._instantiate_and_run(transform, tree)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/codemod/_command.py", line 58, in _instantiate_and_run
    return inst.transform_module(tree)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/codemod/_codemod.py", line 108, in transform_module
    return self.transform_module_impl(tree_with_metadata)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/codemod/_visitor.py", line 32, in transform_module_impl
    return tree.visit(self)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/_nodes/module.py", line 91, in visit
    result = super(Module, self).visit(visitor)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/_nodes/base.py", line 235, in visit
    leave_result = visitor.on_leave(self, with_updated_children)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/matchers/_visitors.py", line 512, in on_leave
    retval = CSTTransformer.on_leave(self, original_node, updated_node)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/_visitors.py", line 72, in on_leave
    updated_node = leave_func(original_node, updated_node)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/codemod/visitors/_add_imports.py", line 393, in leave_Module
    *[
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/codemod/visitors/_add_imports.py", line 394, in <listcomp>
    parse_statement(
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/_parser/entrypoints.py", line 100, in parse_statement
    result = _parse(
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/_parser/entrypoints.py", line 51, in _parse
    result = parser.parse()
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/_parser/base_parser.py", line 111, in parse
    self._add_token(token)
  File "/home/luzfcb/.cache/pre-commit/repof_bg4wfe/py_env-python3/lib/python3.9/site-packages/libcst/_parser/base_parser.py", line 187, in _add_token
    raise ParserSyntaxError(
libcst._exceptions.ParserSyntaxError: Syntax Error @ 1:7.
Incomplete input. Encountered 'import', but expected '.', '...', or 'NAME'.

from  import datetime
      ^

Failed to codemod /home/luzfcb/projects/big_project/foobar/tests/test_managers.py

To Reproduce

pipx install "django-codemod==1.5.5"

echo "from django.utils import timezone" > foobar.py

djcodemod run --deprecated-in 1.9 --deprecated-in 1.10 --deprecated-in 1.11 --deprecated-in 2.0 --deprecated-in 2.1 --deprecated-in 2.2 foobar.py

Additional context

Fixes #214

What this PR does : Merges test.yml, lint.yml and commitlint.yml into a single file ci.yml.

This is my first ever attempt at dealing with CI, so, do let me know if I need to correct something! Thanks :)

Describe the feature Currently the tests are not included in the PyPI sdist. As a result in order to do packaging for openSUSE at https://build.opensuse.org/package/show/devel:languages:python:django/python-django-codemod , I need to fetch the sdist from Github, which is a few extra steps.

Please can the tests be added to the sdist.

To Reproduce Download sdist from PyPI, untar and see that now tests are present.

Link to Django docs Link to release note section where the change and suitable replacement is mentioned:

https://docs.djangoproject.com/en/dev/releases/3.1/#postgresql-jsonfield

Quote of the relevant paragraph

In case links gets outdated/removed:

PostgreSQL JSONField¶ django.contrib.postgres.fields.JSONField and django.contrib.postgres.forms.JSONField are deprecated in favor of models.JSONField and forms.JSONField.

The undocumented django.contrib.postgres.fields.jsonb.KeyTransform and django.contrib.postgres.fields.jsonb.KeyTextTransform are also deprecated in favor of the transforms in django.db.models.fields.json.

The new JSONFields, KeyTransform, and KeyTextTransform can be used on all supported database backends.

Deprecation timeline

• Deprecated in: 3.1
• Removed in: 4.0

My thought is that this scenario should be covered by djcodemod although the case is not explicitly covered in the release notes for 3.1.

It shows up as a deprecation when doing checks but djcodemod doesn't fix:

mint_develop_django | pool.Pool.properties: (fields.W904) django.contrib.postgres.fields.JSONField is deprecated. Support for it (except in historical migrations) will be removed in Django 4.0.
mint_develop_django |   HINT: Use django.db.models.JSONField instead.

The offending import in models.py is:

from django.contrib.postgres.fields.jsonb import JSONField

Probably an artifact of the way Pycharm offers imports.

Fixes #207.

$ djcodemod run --help
Usage: djcodemod run [OPTIONS] PATH...

  Automatically fixes deprecations removed Django deprecations.

  This command takes the path to target as argument and a version of Django
  to select code modifications to apply.

Options:
  --removed-in VERSION            The version of Django where feature are
                                  removed.

  --deprecated-in VERSION         The version of Django where deprecations
                                  started.

  --codemod (see `djcodemod list`)
                                  Choose a specific codemod to run. Can be
                                  repeated.

  --help                          Show this message and exit.

We have 3 workflows that run during CI at the moment: test.yml, lint.yml and commitlint.yml. We could simplify this by merging them into a single ci.yml workflow, with 3 separate jobs test, lint and commitlint.

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | rich | <13 -> <14 | | | | |

Release Notes

Configuration

📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.

• [ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Mend Renovate. View repository job log here.

SUSE Tumbleweed has updated django to 4.0, and while fixing django packages I've noticed a lot of packages are failing with this. e.g. https://build.opensuse.org/package/show/home:jayvdb:branches:devel:languages:python:django/python-django-avatar

https://docs.djangoproject.com/en/dev/releases/4.0/

See Features deprecated in 3.1 for details on these changes, including how to remove usage of these features. .. The providing_args argument for django.dispatch.Signal is removed.

Deprecation timeline

• Deprecated in: 3.1
• Removed in: 4.0

Is your feature request related to a problem? Please describe. When running djcodemod in a build worker, it would be useful to show the changes it made so the logs record the transformations made, and also so it is easy to copy the diffs to submit upstream.

This is especially useful when working from tarballs, as there is no VCS to be able to generate diffs.

Describe the solution you'd like A disabled-by-default flag like --show-diffs which emits the changes which are being written to disk as patch-compatible stdout .

Additional context e.g. it takes a couple of secs to add `djcodemod run --removed-in 4.0 test_project/ to the build scripts for https://build.opensuse.org/package/show/home:jayvdb:branches:devel:languages:python:django/python-django-debreach , to allow me to provide a more detailed issue at https://github.com/lpomfrey/django-debreach/issues/21

But I really should check the diffs to make sure djcodemod is not introducing bugs in the process, and if there were diffs I could have easily added to the upstream issue an inline patch for the first which djcodemod knows how to fix, which gets them closer to the harder second problem which djcodemod doesnt know how to solve.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• [ ] fix(deps): update dependency rich to v13

Detected dependencies

• [ ] Check this box to trigger a request for Renovate to run again on this repository