Speed up Sphinx builds by selectively removing toctrees from some pages

Overview

Remove toctrees from Sphinx pages

Improve your Sphinx build time by selectively removing TocTree objects from pages. This is useful if your documentation uses auto-generated API documentation, which generates a lot of stub pages

This extension can be used to remove the sidebar links for just the pages you specify, speed up the build considerably.

Who is this for?

This package is for maintainers that use Sphinx and have really large API documentation (or for some other reason, have a ton of nested pages). If you use a Sphinx theme that contains the entire Table of Contents on every page (e.g., any theme that has "collapsable" sidebar sections), this will slow things down considerably. Use this theme to speed up your builds.

Install

Install the extension via pip:

$ pip install git+https://github.com/executablebooks/sphinx-remove-toctree

and activate it by adding it to your Sphinx conf.py file:

extensions = ["sphinx_remove_toctree"]

Use

In conf.py, provide a list of glob-like paths relative to your documentation root. Each entry should match to pages that should be removed from the sidebar.

For example, the following configuration will remove all pages from the folder api/generated, and the specific page subfolder/page_two.rst:

remove_toctrees_from = ["api/generated/*", "subfolder/page_two.rst"]

This is particularly useful in combination with the autosummary directive, which tends to generate a ton of stub-pages that slows things down.

If you have the following autosummary directive in a page at myfolder/page1.rst:

.. autosummary: datetime.datetime
   :toctree: api_gen

This will generate stub-pages in a myfolder/api_gen/ folder. To remove each of these pages from your sidebar, you would configure this extension like so:

remove_toctrees_from = ["myfolder/api_gen/*"]

Try it with this documentation

This extension doesn't have a hosted documentation page, but there is one in the docs/ folder of this repository. You can use that folder to preview this extension in action.

How this works

Sphinx keeps track of toctree objects to represent the structure of your documentation. These exist in the Sphinx environment object, at env.tocs. There are two places in the build where this is relevant here:

  • Early in the build, Sphinx uses these tocs to ensure that files in your documentation are linked somewhere, and will raise warnings if it finds a file that is not in one of the tocs.
  • Later in the build, Sphinx uses these tocs to build the HTML toctree with links to pages in your documentation. If there are many elements in tocs, it will take a long time to resolve all of these links!

This extension runs after the first step, but before the second step. It removes all the toctree objects that you specify, so that no warnings are raised about missing files, but they are removed from the sidebar and don't slow down your build.

You might also like...
Sphinx Bootstrap Theme

Sphinx Bootstrap Theme This Sphinx theme integrates the Bootstrap CSS / JavaScript framework with various layout options, hierarchical menu navigation

A powerful Sphinx changelog-generating extension.

What is Releases? Releases is a Python (2.7, 3.4+) compatible Sphinx (1.8+) extension designed to help you keep a source control friendly, merge frien

📖  Generate markdown API documentation from Google-style Python docstring. The lazy alternative to Sphinx.
📖 Generate markdown API documentation from Google-style Python docstring. The lazy alternative to Sphinx.

lazydocs Generate markdown API documentation for Google-style Python docstring. Getting Started • Features • Documentation • Support • Contribution •

Seamlessly integrate pydantic models in your Sphinx documentation.
Seamlessly integrate pydantic models in your Sphinx documentation.

Seamlessly integrate pydantic models in your Sphinx documentation.

python package sphinx template

python-package-sphinx-template python-package-sphinx-template

epub2sphinx is a tool to convert epub files to ReST for Sphinx
epub2sphinx is a tool to convert epub files to ReST for Sphinx

epub2sphinx epub2sphinx is a tool to convert epub files to ReST for Sphinx. It uses Pandoc for converting HTML data inside epub files into ReST. It cr

A `:github:` role for Sphinx

sphinx-github-role A github role for Sphinx. Usage Basic usage MyST: :caption: index.md See {github}`astrojuanlu/sphinx-github-role#1`. reStructuredT

This is a template (starter kit) for writing Maturity Work with Sphinx / LaTeX at Collège du Sud

sphinx-tm-template Ce dépôt est un template de base utilisable pour écrire ton travail de maturité dans le séminaire d'informatique du Collège du Sud.

An MkDocs plugin to export content pages as PDF files

MkDocs PDF Export Plugin An MkDocs plugin to export content pages as PDF files The pdf-export plugin will export all markdown pages in your MkDocs rep

Comments
  • Feedback for the package

    Feedback for the package

    This is a quick issue for feedback on the UX of this package.

    It would be great if folks that have lots of API docs could try the following:

    1. Read the instructions / usage at https://github.com/executablebooks/sphinx-remove-toctrees#readme
    2. Install the extension and activate it
    3. Configure it to suppress the API docs you want out of your sidebar
    4. See how it goes and report back!

    Questions to answer:

    • Did the UX and config make sense?
    • Is this extension explained well enough?
    • Is it named and described properly?

    cc @jorisvandenbossche @rgommers @rossbar @stefanv @jrbourbeau in case any want to give this a shot w/ their docs.

    also cc @pradyunsg and @humitos because I think this extension might be useful for the Furo / RTD themes, as it is handy for any theme that uses collapse=False.

    opened by choldgraf 17
  • 0.0.3: pytest is failing and shows warnings warnings

    0.0.3: pytest is failing and shows warnings warnings

    Describe the bug

    I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

    • python3 -sBm build -w --no-isolation
    • because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
    • install .whl file in </install/prefix>
    • run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

    Here is pytest output:

    + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-remove-toctrees-0.0.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-sphinx-remove-toctrees-0.0.3-2.fc35.x86_64/usr/lib/python3.8/site-packages
    + /usr/bin/pytest -ra
    =========================================================================== test session starts ============================================================================
    platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
    rootdir: /home/tkloczko/rpmbuild/BUILD/sphinx-remove-toctrees-0.0.3
    plugins: timeout-2.1.0, vcr-1.0.2
    collected 1 item
    
    sphinx_remove_toctrees/tests/test_build.py .                                                                                                                         [100%]
    
    ============================================================================= warnings summary =============================================================================
    sphinx_remove_toctrees/tests/test_build.py::test_build_html
      /home/tkloczko/rpmbuild/BUILD/sphinx-remove-toctrees-0.0.3/sphinx_remove_toctrees/tests/test_build.py:21: GuessedAtParserWarning: No parser was explicitly specified, so I'm using the best available HTML parser for this system ("lxml"). This usually isn't a problem, but if you run this code on another system, or in a different virtual environment, it may use a different parser and behave differently.
    
      The code that caused this warning is on line 21 of the file /home/tkloczko/rpmbuild/BUILD/sphinx-remove-toctrees-0.0.3/sphinx_remove_toctrees/tests/test_build.py. To get rid of this warning, pass the additional argument 'features="lxml"' to the BeautifulSoup constructor.
    
        index = BeautifulSoup(index.read_text())
    
    -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
    ======================================================================= 1 passed, 1 warning in 0.77s =======================================================================
    

    Reproduce the bug

    N/A

    List your environment

    N/A

    bug 
    opened by kloczek 3
Releases(v0.0.3)
Owner
Executable Books
An open collaboration to create executable books with Jupyter
Executable Books
Parser manager for parsing DOC, DOCX, PDF or HTML files

Parser manager Description Parser gets PDF, DOC, DOCX or HTML file via API and saves parsed data to the database. Implemented in Ruby 3.0.1 using Acti

Эдем 4 Dec 04, 2021
Credit EDA Case Study Using Python

This case study aims to identify patterns which indicate if a client has difficulty paying their installments which may be used for taking actions such as denying the loan, reducing the amount of loa

Purvi Padliya 1 Jan 14, 2022
Get link preview of a website.

Preview Link You may have seen a preview of a link with a title, image, domain, and description when you share a link on social media. This preview ha

SREEHARI K.V 8 Jan 08, 2023
Word document generator with python

In this study, real world data is anonymized. The content is completely different, but the structure is the same. It was a script I prepared for the backend of a work using UiPath.

Ezgi Turalı 3 Jan 30, 2022
API Documentation for Python Projects

API Documentation for Python Projects. Example pdoc -o ./html pdoc generates this website: pdoc.dev/docs. Installation pip install pdoc pdoc is compat

mitmproxy 1.4k Jan 07, 2023
Reproducible Data Science at Scale!

Pachyderm: The Data Foundation for Machine Learning Pachyderm provides the data layer that allows machine learning teams to productionize and scale th

Pachyderm 5.7k Dec 29, 2022
Second version of SQL-PYTHON-Practicas

SQLite-Python Acerca de | Autor Sobre el repositorio Segunda version de SQL-PYTHON-Practicas 💻 Tecnologias Visual Studio Code Python SQLite3 📖 Requi

1 Jan 06, 2022
Python Tool to Easily Generate Multiple Documents

Python Tool to Easily Generate Multiple Documents Running the script doesn't require internet Max Generation is set to 10k to avoid lagging/crashing R

2 Apr 27, 2022
Version bêta d'un système pour suivre les prix des livres chez Books to Scrape,

Version bêta d'un système pour suivre les prix des livres chez Books to Scrape, un revendeur de livres en ligne. En pratique, dans cette version bêta, le programme n'effectuera pas une véritable surv

Mouhamed Dia 1 Jan 06, 2022
Resource hub for Obsidian resources.

Obsidian Community Vault Welcome! This is an experimental vault that is maintained by the Obsidian community. For best results we recommend downloadin

Obsidian Community 320 Jan 02, 2023
EasyMultiClipboard - Python script written to handle more than 1 string in clipboard

EasyMultiClipboard - Python script written to handle more than 1 string in clipboard

WVlab 1 Jun 18, 2022
Pydantic model generator for easy conversion of JSON, OpenAPI, JSON Schema, and YAML data sources.

datamodel-code-generator This code generator creates pydantic model from an openapi file and others. Help See documentation for more details. Supporte

Koudai Aono 1.3k Dec 29, 2022
Main repository for the Sphinx documentation builder

Sphinx Sphinx is a tool that makes it easy to create intelligent and beautiful documentation for Python projects (or other documents consisting of mul

5.1k Jan 04, 2023
A simple malware that tries to explain the logic of computer viruses with Python.

Simple-Virus-With-Python A simple malware that tries to explain the logic of computer viruses with Python. What Is The Virus ? Computer viruses are ma

Xrypt0 6 Nov 18, 2022
epub2sphinx is a tool to convert epub files to ReST for Sphinx

epub2sphinx epub2sphinx is a tool to convert epub files to ReST for Sphinx. It uses Pandoc for converting HTML data inside epub files into ReST. It cr

Nihaal 8 Dec 15, 2022
Fun interactive program to sort a list :)

LHD-Build-Sort-a-list Fun interactive program to sort a list :) Inspiration LHD Build Write a script to sort a list. What it does It is a menu driven

Ananya Gupta 1 Jan 15, 2022
DocumentPy is a Python application that runs in a command-line interface environment, made for creating HTML documents.

DocumentPy DocumentPy is a Python application that runs in a command-line interface environment, made for creating HTML documents. Usage DocumentPy, a

Lotus 0 Jul 15, 2021
Python Eacc is a minimalist but flexible Lexer/Parser tool in Python.

Python Eacc is a parsing tool it implements a flexible lexer and a straightforward approach to analyze documents.

Iury de oliveira gomes figueiredo 60 Nov 16, 2022
Autolookup GUI Plugin for Plover

Word Tray for Plover Word Tray is a GUI plugin that automatically looks up efficient outlines for words that start with the current input, much like a

Kathy 3 Jun 08, 2022
Spin-off Notice: the modules and functions used by our research notebooks have been refactored into another repository

Fecon235 - Notebooks for financial economics. Keywords: Jupyter notebook pandas Federal Reserve FRED Ferbus GDP CPI PCE inflation unemployment wage income debt Case-Shiller housing asset portfolio eq

Adriano 825 Dec 27, 2022