Simple reproduction of connection leak with celery/django/gevent

Related tags

Djangoredis-leak
Overview

Redis connection leak with celery/django/gevent

Reproduces celery issue at https://github.com/celery/celery/issues/6819 using gevented django web server and redis cache.

Some more good discussion here: https://github.com/redis/redis-py/issues/1495

Setup

  1. pip install -r requirements.txt
  2. Run server: gunicorn -k gevent redis_leak.wsgi:application
  3. Run celery: celery -A redis_leak.celery worker --pool gevent --loglevel=info

Make a ton of requests

First run python load.py - this will make requests to our django webserver to a view that executes and waits for the result from a debug celery task. This will make these requests in a tight loop until you kill the process. Keep this running while you monitor your redis connections

Meaure connections

  • redis-cli info Clients shows "connected_clients"

  • while /bin/true; do date --iso=seconds; redis-cli -c -h localhost -p 6379 CLIENT LIST | wc -l; sleep 2; done;

You'll hopefully find these connections are growing as your requests pile up. You'll also find these connections remain after killing your request loop. If you kill your gunicorn+gevent webserver they all go away.

RedisTimeSeries python client

redistimeseries-py Deprecation notice As of redis-py 4.0.0 this library is deprecated. It's features have been merged into redis-py. Please either ins

98 Dec 08, 2022
A quick way to add React components to your Django templates.

Django-React-Templatetags This django library allows you to add React (16+) components into your django templates. Features Include react components u

Fröjd Agency 408 Jan 08, 2023
Probably the best abstract model / admin for your tree based stuff.

django-treenode Probably the best abstract model / admin for your tree based stuff. Features Fast - get ancestors, children, descendants, parent, root

Fabio Caccamo 360 Jan 05, 2023
It's the assignment 1 from the Python 2 course, that requires a ToDoApp with authentication using Django

It's the assignment 1 from the Python 2 course, that requires a ToDoApp with authentication using Django

0 Jan 20, 2022
This website serves as an online database (hosted via SQLLite) for fictional businesses in the area to store contact information (name, email, phone number, etc.) for fictional customers.

Django-Online-Business-Database-Project this project is still in progress Overview of Website This website serves as an online database (hosted via SQ

1 Oct 30, 2021
MAC address Model Field & Form Field for Django apps

django-macaddress MAC Address model and form fields for Django We use netaddr to parse and validate the MAC address. The tests aren't complete yet. Pa

49 Sep 04, 2022
Ugly single sign-on for django projects only

django-usso Ugly single sign-on for django projects only. Do you have many django apps with different users? Do you want to use only one of those apps

Erwin Feser 1 Mar 01, 2022
A multiprocessing distributed task queue for Django

A multiprocessing distributed task queue for Django Features Multiprocessing worker pool Asynchronous tasks Scheduled, cron and repeated tasks Signed

Ilan Steemers 1.7k Jan 03, 2023
Django Livre Bank

Django Livre Bank Projeto final da academia Construdelas. API de um banco fictício com clientes, contas e transações. Integrantes da equipe Bárbara Sa

Cecília Costa 3 Dec 22, 2021
Money fields for Django forms and models.

django-money A little Django app that uses py-moneyed to add support for Money fields in your models and forms. Django versions supported: 1.11, 2.1,

1.4k Jan 06, 2023
Django application and library for importing and exporting data with admin integration.

django-import-export django-import-export is a Django application and library for importing and exporting data with included admin integration. Featur

2.6k Dec 26, 2022
Twitter Bootstrap for Django Form - A simple Django template tag to work with Bootstrap

Twitter Bootstrap for Django Form - A simple Django template tag to work with Bootstrap

tzangms 557 Oct 19, 2022
Forgot password functionality build in Python / Django Rest Framework

Password Recover Recover password functionality with e-mail sender usign Django Email Backend How to start project. Create a folder in your machine Cr

alexandre Lopes 1 Nov 03, 2021
A better and faster multiple selection widget with suggestions

django-searchable-select A better and faster multiple selection widget with suggestions for Django This project is looking for maintainers! Please ope

Andrew Dunai 105 Oct 22, 2022
Getdp-project - A Django-built web app that generates a personalized banner of events to come

getdp-project https://get-my-dp.herokuapp.com/ A Django-built web app that gener

CODE 4 Aug 01, 2022
With Django Hijack, admins can log in and work on behalf of other users without having to know their credentials.

Django Hijack With Django Hijack, admins can log in and work on behalf of other users without having to know their credentials. Docs 3.x docs are avai

1.2k Jan 05, 2023
This is a repository for collecting global custom management extensions for the Django Framework.

Django Extensions Django Extensions is a collection of custom extensions for the Django Framework. Getting Started The easiest way to figure out what

Django Extensions 6k Dec 26, 2022
Strict separation of config from code.

Python Decouple: Strict separation of settings from code Decouple helps you to organize your settings so that you can change parameters without having

Henrique Bastos 2.3k Jan 04, 2023
A django model and form field for normalised phone numbers using python-phonenumbers

django-phonenumber-field A Django library which interfaces with python-phonenumbers to validate, pretty print and convert phone numbers. python-phonen

Stefan Foulis 1.3k Dec 31, 2022
Utilities for implementing a modified pre-order traversal tree in django.

django-mptt Utilities for implementing Modified Preorder Tree Traversal with your Django Models and working with trees of Model instances. Project hom

2.7k Jan 01, 2023