Simple reproduction of connection leak with celery/django/gevent

Related tags

Djangoredis-leak
Overview

Redis connection leak with celery/django/gevent

Reproduces celery issue at https://github.com/celery/celery/issues/6819 using gevented django web server and redis cache.

Some more good discussion here: https://github.com/redis/redis-py/issues/1495

Setup

  1. pip install -r requirements.txt
  2. Run server: gunicorn -k gevent redis_leak.wsgi:application
  3. Run celery: celery -A redis_leak.celery worker --pool gevent --loglevel=info

Make a ton of requests

First run python load.py - this will make requests to our django webserver to a view that executes and waits for the result from a debug celery task. This will make these requests in a tight loop until you kill the process. Keep this running while you monitor your redis connections

Meaure connections

  • redis-cli info Clients shows "connected_clients"

  • while /bin/true; do date --iso=seconds; redis-cli -c -h localhost -p 6379 CLIENT LIST | wc -l; sleep 2; done;

You'll hopefully find these connections are growing as your requests pile up. You'll also find these connections remain after killing your request loop. If you kill your gunicorn+gevent webserver they all go away.

Django-shared-app-isolated-databases-example - Django - Shared App & Isolated Databases

Django - Shared App & Isolated Databases An app that demonstrates the implementa

Ajai Danial 5 Jun 27, 2022
A simple REST API to manage postal addresses, written in Python/Django.

A simple REST API to manage postal addresses, written in Python/Django.

Attila Bagossy 2 Feb 14, 2022
✋ Auto logout a user after specific time in Django

django-auto-logout Auto logout a user after specific time in Django. Works with Python 🐍 ≥ 3.7, Django 🌐 ≥ 3.0. ✔️ Installation pip install django-a

Georgy Bazhukov 21 Dec 26, 2022
An airlines clone website with django

abc_airlines is a clone website of an airlines system the way it works is that first you add flights to the website then the users can search flights

milad 1 Nov 16, 2021
A drop-in replacement for django's ImageField that provides a flexible, intuitive and easily-extensible interface for quickly creating new images from the one assigned to the field.

django-versatileimagefield A drop-in replacement for django's ImageField that provides a flexible, intuitive and easily-extensible interface for creat

Jonathan Ellenberger 490 Dec 13, 2022
Stream Framework is a Python library, which allows you to build news feed, activity streams and notification systems using Cassandra and/or Redis. The authors of Stream-Framework also provide a cloud service for feed technology:

Stream Framework Activity Streams & Newsfeeds Stream Framework is a Python library which allows you to build activity streams & newsfeeds using Cassan

Thierry Schellenbach 4.7k Jan 02, 2023
Run Django tests with testcontainers.

django-rdtwt (Run Django Tests With Testcontainers) This targets users who wish to forget setting up a database for tests. There's no manually startin

2 Jan 09, 2022
A collection of models, views, middlewares, and forms to help secure a Django project.

Django-Security This package offers a number of models, views, middlewares and forms to facilitate security hardening of Django applications. Full doc

SD Elements 258 Jan 03, 2023
xsendfile etc wrapper

Django Sendfile This is a wrapper around web-server specific methods for sending files to web clients. This is useful when Django needs to check permi

John Montgomery 476 Dec 01, 2022
Django Simple Spam Blocker is blocking spam by regular expression.

Django Simple Spam Blocker is blocking spam by regular expression.

Masahiko Okada 23 Nov 29, 2022
Getdp-project - A Django-built web app that generates a personalized banner of events to come

getdp-project https://get-my-dp.herokuapp.com/ A Django-built web app that gener

CODE 4 Aug 01, 2022
Simple application TodoList django with ReactJS

Django & React Django We basically follow the Django REST framework quickstart guide here. Create backend folder with a virtual Python environment: mk

Flavien HUGS 2 Aug 07, 2022
📊📈 Serves up Pandas dataframes via the Django REST Framework for use in client-side (i.e. d3.js) visualizations and offline analysis (e.g. Excel)

Django REST Pandas Django REST Framework + pandas = A Model-driven Visualization API Django REST Pandas (DRP) provides a simple way to generate and se

wq framework 1.2k Jan 01, 2023
This repository contains django library management system project.

Library Management System Django ** INSTALLATION** First of all install python on your system. Then run pip install -r requirements.txt to required se

whoisdinanath 1 Dec 26, 2022
PEP-484 type hints bindings for the Django web framework

mypy-django Type stubs to use the mypy static type-checker with your Django projects This project includes the PEP-484 compatible "type stubs" for Dja

Machinalis 223 Jun 17, 2022
Django/Jinja template indenter

DjHTML A pure-Python Django/Jinja template indenter without dependencies. DjHTML is a fully automatic template indenter that works with mixed HTML/CSS

Return to the Source 378 Jan 01, 2023
A Powerful HTML white space remover for Django

HTML Whitespace remover for Django Introduction : A powerful tool to optimize Django rendered templates Why use "django_stip_whitespace" ? Adds line b

3 Jan 01, 2022
A simple app that provides django integration for RQ (Redis Queue)

Django-RQ Django integration with RQ, a Redis based Python queuing library. Django-RQ is a simple app that allows you to configure your queues in djan

RQ 1.6k Jan 06, 2023
Use minify-html, the extremely fast HTML + JS + CSS minifier, with Django.

django-minify-html Use minify-html, the extremely fast HTML + JS + CSS minifier, with Django. Requirements Python 3.8 to 3.10 supported. Django 2.2 to

Adam Johnson 60 Dec 28, 2022
Send push notifications to mobile devices through GCM or APNS in Django.

django-push-notifications A minimal Django app that implements Device models that can send messages through APNS, FCM/GCM and WNS. The app implements

Jazzband 2k Dec 26, 2022