Django query profiler - one profiler to rule them all. Shows queries, detects N+1 and gives recommendations on how to resolve them

Overview

Django Query Profiler

https://travis-ci.com/django-query-profiler/django-query-profiler.svg?branch=master https://codecov.io/gh/django-query-profiler/django-query-profiler/branch/master/graph/badge.svg?token=1Cv7WsOi2W https://readthedocs.org/projects/django-query-profiler/badge/?version=latest https://img.shields.io/pypi/djversions/django-query-profiler

This is a query profiler for Django applications, for helping developers answer the question "My Django code/page/API is slow, How do I find out why?"

Below are some of the features of the profiler:

  1. Shows code paths making N+1 sql calls: Shows the sql with stack_trace which is making N+1 calls, along with sql count
  2. Shows the proposed solution: If the solution to reduce sql is to simply apply a select_related or a prefetch_related, this is highlighted as a suggestion
  3. Shows exact sql duplicates: Count of the queries where (sql, parameters) is exactly the same. This is the kind of sql where implementing a query cache would help
  4. Flame Graph visualisation: Collects all the stack traces together to allow quickly identifying which area(s) of code is causing the load to the database
  5. Command line or chrome plugin: The profiler can be called from command line via context manager, or can be invoked via a middleware, and output shown in a chrome plugin
  6. Super easy to configure in any application: The only changes are in settings.py file and in urls.py file

This is the repo for the chrome plugin

Requirements

This works with any version of django >= 2.0, and running on python >= 3.6

Profiler in Action

as a chrome plugin

This image shows how the chrome plugin would display profiled data, once it is configured & installed

https://raw.githubusercontent.com/django-query-profiler/django-query-profiler/master/docs/_static/django_query_profiler_in_action.gif

on command line

See this file in the PR to see how to use the context manager, and how easy it is to find performance issues :-)

The output of Django query profiler is same for the command line or the chrome plugin. In fact, chrome plugin displays the output set by the middleware - which is just a plain wrapper around context manager.

Getting Started

installation

The simplest way to getting started is to install the django query profiler from pip, and get the chrome plugin from chrome web store.

Python package:

pip install django-query-profiler

Chrome Plugin:

Download from chrome webstore

This is covered in detail in the installation section in the docs

configuration:

This configuration is when we want to use the profiler along with the chrome plugin. If we want to just use it on the command line, the configuration is much more simpler (two lines of change to settings.py file) - that is covered in the docs

settings.py:

from django_query_profiler.settings import *

INSTALLED_APPS = (
    ...
    'django_query_profiler',
    ...
)

MIDDLEWARE = (
    ...
     # Request and all middleware that come after our middleware, would be profiled
    'django_query_profiler.client.middleware.QueryProfilerMiddleware',
    ...
)

DATABASES = (
    ...
    # Adding django_query_profiler as a prefix to your ENGINE setting
    # Assuming old ENGINE was "django.db.backends.sqlite3", this would be the new one
    "ENGINE": "django_query_profiler.django.db.backends.sqlite3",
)

urls.py:

# Add this line to existing urls.py
path('django_query_profiler/', include('django_query_profiler.client.urls'))

See this PR on how to configure this in your application, and how the plugin is going to look like after your configuration

https://raw.githubusercontent.com/django-query-profiler/django-query-profiler/master/docs/_static/chrome_plugin.png

This is covered in detail in the configuration instructions section in the docs

How the profiler works

This is also covered in detail in the documentation at how the profiler works section in the docs, along with how the code is organized.

The docs also contain references to various links which helped us to lear about internals of Django, and to various projects which helped us to learn on how to add hooks when Django executes a query

Choosing Profiler levels

We have two levels of profiler, and each of them have a different overhead. The two levels are:

  1. QUERY_SIGNATURE: This is the mode where we capture the query as well as the stack-trace. This mode figures out the N+1 code paths and also tells us the proposed solution
  2. QUERY: This is the mode where we just capture queries, and not the stack-trace

On an average, QUERY_SIGNATURE level adds an overhead of 1 millisecond per 7 queries, and QUERY_SIGNATURE adds an overhead of 1 millisecond per 25 queries.

It is simple to change the profiler level for all the requests, or can be configured per request. This is covered in the choosing profiler level section of the docs

Customizing the profiler

We have tried to make the profiler customizable by providing hooks at various points. Some of the use cases are covered here in the customizing the defaults section in docs.

We plan to add more hooks for customizing the profiler as we gather more feedback from real world use cases.

For contributors

https://img.shields.io/badge/PRs-welcome-brightgreen.svg?style=flat-square

The django query profiler is released under the BSD license, like Django itself.

If you like it, please consider contributing! The docs cover everything from how to setup locally, to how the code is organized to running tests.

Documentation

Full documentation is available at readthedocs

Owner
Django Query Profiler
Contact us at [email protected]
Django Query Profiler
pdm-django: Django command shortcuts for PDM

pdm-django: Django command shortcuts for PDM A plugin that gives you command shortcuts for developing with PDM. pdm run python manage.py runserver -

Neutron Sync 2 Aug 11, 2022
A pickled object field for Django

django-picklefield About django-picklefield provides an implementation of a pickled object field. Such fields can contain any picklable objects. The i

Gintautas Miliauskas 167 Oct 18, 2022
This repository contains django library management system project.

Library Management System Django ** INSTALLATION** First of all install python on your system. Then run pip install -r requirements.txt to required se

whoisdinanath 1 Dec 26, 2022
Dynamic Django settings.

Constance - Dynamic Django settings A Django app for storing dynamic settings in pluggable backends (Redis and Django model backend built in) with an

Jazzband 1.5k Jan 07, 2023
Helps working with singletons - things like global settings that you want to edit from the admin site.

Django Solo +---------------------------+ | | | | | \ | Django Solo helps

Sylvain Toé 726 Jan 08, 2023
Simple tagging for django

django-taggit This is a Jazzband project. By contributing you agree to abide by the Contributor Code of Conduct and follow the guidelines. django-tagg

Jazzband 3k Jan 02, 2023
The new Python SDK for Sentry.io

Bad software is everywhere, and we're tired of it. Sentry is on a mission to help developers write better software faster, so we can get back to enjoy

Sentry 1.4k Jan 05, 2023
Redia Cache implementation in django.

django-redis Recipe APP Simple Recipe app which shows different kinds off recipe to the user. Why Cache ? Accessing data from cache is much faster tha

Avinash Alanjkar 1 Sep 21, 2022
Imparare Django ricreando un sito facsimile a quello Flask

SitoPBG-Django Imparare Django ricreando un sito facsimile a quello Flask Note di utilizzo Necessita la valorizzazione delle seguenti variabili di amb

Mario Nardi 1 Dec 08, 2021
Learn Python and the Django Framework by building a e-commerce website

The Django-Ecommerce is an open-source project initiative and tutorial series built with Python and the Django Framework.

Very Academy 275 Jan 08, 2023
📝 Sticky Notes in Django admin

django-admin-sticky-notes Share notes between superusers. Installation Install via pip: pip install django_admin_sticky_notes Put django_admin_sticky_

Dariusz Choruży 7 Oct 06, 2021
Tools to easily create permissioned CRUD endpoints in graphene-django.

graphene-django-plus Tools to easily create permissioned CRUD endpoints in graphene-django. Install pip install graphene-django-plus To make use of ev

Zerosoft 74 Aug 09, 2022
🔥 Campus-Run Django Server🔥

🏫 Campus-Run Campus-Run is a 3D racing game set on a college campus. Designed this service to comfort university students who are unable to visit the

Youngkwon Kim 1 Feb 08, 2022
Automatic class scheduler for Texas A&M written with Python+Django and React+Typescript

Rev Registration Description Rev Registration is an automatic class scheduler for Texas A&M, aimed at easing the process of course registration by gen

Aggie Coding Club 21 Nov 15, 2022
The magical reactive component framework for Django ✨

Unicorn The magical full-stack framework for Django ✨ Unicorn is a reactive component framework that progressively enhances a normal Django view, make

Adam Hill 1.4k Jan 05, 2023
Logan is a toolkit for building standalone Django applications

Logan Logan is a toolkit for running standalone Django applications. It provides you with tools to create a CLI runner, manage settings, and the abili

David Cramer 206 Jan 03, 2023
Simple yet powerful and really extendable application for managing a blog within your Django Web site.

Django Blog Zinnia Simple yet powerful and really extendable application for managing a blog within your Django Web site. Zinnia has been made for pub

Julien Fache 2.1k Dec 24, 2022
The best way to have DRY Django forms. The app provides a tag and filter that lets you quickly render forms in a div format while providing an enormous amount of capability to configure and control the rendered HTML.

django-crispy-forms The best way to have Django DRY forms. Build programmatic reusable layouts out of components, having full control of the rendered

4.6k Jan 07, 2023
A simple trivia quizzz web app made using django

Trivia Quizzz A simple trivia quizzz web app made using django Demo http://triviaquizzz.herokuapp.com/ & https://triviaquiz.redcrypt.xyz Features Goog

Rachit Khurana 2 Feb 10, 2022
Alt1-compatible widget host for RuneScape 3

RuneKit Alt1-compatible toolbox for RuneScape 3, for Linux and macOS. Compatibility macOS installation guide Running This project use Poetry as packag

Manatsawin Hanmongkolchai 75 Nov 28, 2022