Create docsets for Dash.app-compatible API browser.

Overview

doc2dash: Create Docsets for Dash.app and Clones

Documentation Status CI Status Test Coverage Code style: black

doc2dash is an MIT-licensed extensible Documentation Set generator intended to be used with the Dash.app API browser for macOS or one of its many free clones for all relevant platforms.

If you’ve never heard of Dash.app and its likes, you’re missing out: together with doc2dash it’s all your API documentation at your fingertips – even when you're offline!

doc2dash’s documentation lives at Read the Docs, the code on GitHub. It’s tested on Python 3.8+ (the fully functional 2.x release line on PyPI also supports Python 2.7, 3.4–3.6, and PyPy). Both Linux and macOS are tested although certain features are only available on macOS.

Comments
  • Intersphinx

    Intersphinx

    Use Sphinx’s own intersphinx files and APIs to get all symbol names. That’s both more robust and gives us a better introspection into the actual types of symbols.

    Todos:

    • [x] front page as reported in https://github.com/hynek/doc2dash/issues/27#issuecomment-50266807
    • [x] bring coverage back to 100%

    Should fix #27 and #25.

    opened by hynek 14
  • Make InterSphinxParser extensible

    Make InterSphinxParser extensible

    Partial fix for #57

    • allow INV_TO_TYPE mapping to be overridden via convert_type method
    • allow ParserEntry creation to be overridden with create_entry method
    opened by jnothman 12
  • Trying to build docset for sagemath results in error

    Trying to build docset for sagemath results in error

    Hi, thanks for this tool. I have installed sagemath through nix and it includes documentation using sphinx. I am trying to build a docset for it using

    doc2dash -n Sage -f -d ~/.local/share/dashdocs/ -i ~/Downloads/sage_logo_new_hc-nq8.png /nix/store/150yjwckyscc2kisgx93fs40cwk5wwdp-sagedoc-9.6/share/doc/sage/html/en/reference
    

    However this fails with

    PermissionError: [Errno 13] Permission denied: '.buildinfo'
    

    The .buildinfo files seem to be part of many sub-directories of the documentation directory and the look like this,

    # Sphinx build info version 1
    # This file hashes the configuration used when building these files. When it is not found, a full rebuild will be done.
    config: f164681b73cecad2fb4d00961dc5623f
    tags: 645f666f9bcd5a90fca523b33c5a78b7
    

    I assume these are for nix (I don't know much, sagemath is the only thing I use nix for). Is it possible to ignore these files somehow? Or for doc2dash to continue despite such errors and generate docset based on files it can successfully consume?

    opened by aikrahguzar 10
  • Cannot detect Gevent's documentation format

    Cannot detect Gevent's documentation format

    Gevent's document is generated by Sphinx, but I ran: doc2dash -n 'Py2_gevent' -I ./gevent/doc/_build/html/contents.html ./gevent/doc/_build/html/

    and got the message: "./gevent/doc/_build/html/" does not contain a known documentation format.

    opened by wenLiangcan 10
  • Not adding table of contents metadata with latest pydoctor

    Not adding table of contents metadata with latest pydoctor

    doc2dash isn't adding the table of contents metadata with docs that are generated using the latest pydoctor. Happens with the Twisted docs. No errors are shown.

    opened by Kapeli 9
  • Q: how to avoid to generate entries point to not exists file.

    Q: how to avoid to generate entries point to not exists file.

    Hi, I'm trying to use doc2dash to generate scikit-learn doc for dash https://github.com/Kapeli/Dash-User-Contributions/pull/3805 https://github.com/Kapeli/Dash-User-Contributions/pull/3812. However, the ci complains about File not found:

    Scikit/scikit-learn.tgz: FAIL ❌
    File not found at "genindex.html" for entry with row id "3058" of "scikit-learn.docset"
    File not found at "py-modindex.html" for entry with row id "3222" of "scikit-learn.docset"
    File not found at "py-modindex.html" for entry with row id "3327" of "scikit-learn.docset"
    [NOTICE] Some pages loaded online resources, which might mean the docset doesn't work offline.
    

    I am asking for advice on how to solve this problem, here are the commands I tried:

    # download scikit-learn docs directly
    wget https://scikit-learn.org/stable/_downloads/scikit-learn-docs.zip
    unzip -d scikit-learn-docs scikit-learn-docs.zip
    
    # convert to dash
    doc2dash --index-page documentation.html --enable-js -n scikit-learn scikit-learn-docs
    tar --exclude='.DS_Store' -czvf scikit-learn.tgz scikit-learn.docset
    
    opened by tshu-w 8
  • Fix URL encoded filenames

    Fix URL encoded filenames

    I was trying to use doc2dash with the documentation of quimb but the screen show the following error.

    Converting intersphinx docs from "html" to "./html.docset".
    Parsing documentation...
    Added 2,494 index entries.
    Adding table of contents meta data...  [##################################################-------------------------------------------]   53%  00:00:18
    Traceback (most recent call last):
      File "/Users/mofeing/Develop/quimb/.venv/bin/doc2dash", line 8, in <module>
        sys.exit(main())
      File "/Users/mofeing/Develop/quimb/.venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
        return self.main(*args, **kwargs)
      File "/Users/mofeing/Develop/quimb/.venv/lib/python3.8/site-packages/click/core.py", line 782, in main
        rv = self.invoke(ctx)
      File "/Users/mofeing/Develop/quimb/.venv/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/Users/mofeing/Develop/quimb/.venv/lib/python3.8/site-packages/click/core.py", line 610, in invoke
        return callback(*args, **kwargs)
      File "/Users/mofeing/Develop/quimb/.venv/lib/python3.8/site-packages/doc2dash/__main__.py", line 227, in main
        toc.close()
      File "/Users/mofeing/Develop/quimb/.venv/lib/python3.8/site-packages/doc2dash/parsers/utils.py", line 132, in patch_anchors
        patch_files(pbar)
      File "/Users/mofeing/Develop/quimb/.venv/lib/python3.8/site-packages/doc2dash/parsers/utils.py", line 112, in patch_files
        with codecs.open(full_path, mode="r", encoding="utf-8") as fp:
      File "/Users/mofeing/.pyenv/versions/3.8.2/lib/python3.8/codecs.py", line 905, in open
        file = builtins.open(filename, mode, buffering)
    FileNotFoundError: [Errno 2] No such file or directory: './html.docset/Contents/Resources/Documents/calculating%20quantities.html'
    

    Problem was that the repo generates some HTML files with spaces in their names and other files referencing to them encodes their names in URL form. Adding a URL decoding step solves the problem and does not interfere with other filenames.

    opened by mofeing 8
  • Fix html anchor parsing wrong in some docs

    Fix html anchor parsing wrong in some docs

    Some path parse from objects.inv like the one in alembic's would be 'index.html#document-api/ddl#alembic.ddl.mysql.MySQLChangeColumn'. As you see, there is more than one '#' in this path, and it cannot be recognised correctly. That may be the inv file's error. This commit replace the path with head and tail of the original one. It can fix this issue.

    opened by asherascout 8
  • Error when creating docset

    Error when creating docset

    Hello,

    i would like to create a docset for guzzle. But when running the doc2dash command:

    doc2dash _build/html -n guzzle-docs -d /Users/USERNAME/Git/guzzle/docs
    

    i get an error message.

    Converting intersphinx docs from "html" to "/Users/USERNAME/Git/guzzle/docs/guzzle-docs.docset".
    Parsing documentation...
    Added 22 index entries.
    Adding table of contents meta data...  [#################################################################################################################################################################------------------------------------------------------------------------------------------------------------]   60%
    Traceback (most recent call last):
      File "/Users/USERNAME/.virtualenvs/doc2dashtest/bin/doc2dash", line 8, in <module>
        sys.exit(main())
      File "/Users/USERNAME/.virtualenvs/doc2dashtest/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
        return self.main(*args, **kwargs)
      File "/Users/USERNAME/.virtualenvs/doc2dashtest/lib/python3.9/site-packages/click/core.py", line 1055, in main
        rv = self.invoke(ctx)
      File "/Users/USERNAME/.virtualenvs/doc2dashtest/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/Users/USERNAME/.virtualenvs/doc2dashtest/lib/python3.9/site-packages/click/core.py", line 760, in invoke
        return __callback(*args, **kwargs)
      File "/Users/USERNAME/.virtualenvs/doc2dashtest/lib/python3.9/site-packages/doc2dash/__main__.py", line 227, in main
        toc.close()
      File "/Users/USERNAME/.virtualenvs/doc2dashtest/lib/python3.9/site-packages/doc2dash/parsers/utils.py", line 134, in patch_anchors
        patch_files(pbar)
      File "/Users/USERNAME/.virtualenvs/doc2dashtest/lib/python3.9/site-packages/doc2dash/parsers/utils.py", line 114, in patch_files
        with codecs.open(full_path, mode="r", encoding="utf-8") as fp:
      File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/codecs.py", line 905, in open
        file = builtins.open(filename, mode, buffering)
    FileNotFoundError: [Errno 2] No such file or directory: '/Users/USERNAME/Git/guzzle/docs/guzzle-docs.docset/Contents/Resources/Documents/py-modindex.html'
    

    any ideas?

    doc2dash version: 2.4.1

    opened by crydotsnake 7
  • Mypy sphynx documentation not supported ??

    Mypy sphynx documentation not supported ??

    Hi,

    I'm trying to generate a Dash docset for mypy, unfortunatly it doesn't work and I don't understand what I'm doing wrong.

    I cloned the mypy repo I installed doc2dash with pip I go do mypy/docs I type :

    >doc2dash -n mypy .
    

    End result is :

    "/Users/XXXXX/dev/python/mypy/docs" does not contain a known documentation format.
    

    I tried with enabling the -v option, with no more success.

    Any help is welcomed 😃

    opened by keufran 7
  • Cannot import BeautifulSoup documentation

    Cannot import BeautifulSoup documentation

    Doc2Dash seems very cool but unfortunately, I was not able to import BeautifulSoup documentation :S

    Steps to reproduce:

    wget http://www.crummy.com/software/BeautifulSoup/bs4/download/4.3/beautifulsoup4-4.3.2.tar.gz
    tar xzvf beautifulsoup4-4.3.2.tar.gz
    cd beautifulsoup4-4.3.2/doc
    make html
    doc2dash -A build/html -n BeautifulSoup
    

    In Dash, I see "Empty docset". When I open beautifulsoup4-4.3.2/doc/build/html/index.html, the documentation is good. Have I done something wrong ?

    Thanks !

    opened by ptbrowne 7
  • 'Can't find anchor ... in ...' error when compiling docs with doc2dash

    'Can't find anchor ... in ...' error when compiling docs with doc2dash

    When I build the statsmodels docs for Dash, I usually get these types of errors, the 'Can't find anchor ... in ...'. Below are some such examples. Even so, the docs are created, however what I've noticed is that some index entries are repeated, such as the entries for sections (see image below).

    Is this a statsmodels issue, or can it be confronted in doc2dash ?

    Can't find anchor 'statsmodels.genmod.families.links.CLogLog' (EntryType.CLASS) in 'generated/statsmodels.genmod.families.links.CLogLog.html'.

    Can't find anchor 'statsmodels.genmod.families.links.Log' (EntryType.SECTION) in 'generated/statsmodels.genmod.families.links.Log.html'.

    Can't find anchor 'statsmodels.genmod.families.links.CLogLog.deriv' (EntryType.METHOD) in 'generated/statsmodels.genmod.families.links.CLogLog.deriv.html'.

    repeated-entries-in-sections
    opened by capac 3
  • doc2dash Linux binary does not support older operating systems

    doc2dash Linux binary does not support older operating systems

    Here is the current OS where you run pyoxidizer:

    https://github.com/hynek/doc2dash/blob/46bba9b667da609bb38981901dde1e68574d925f/.github/workflows/pyoxidizer.yml#L20

    On GitHub Actions, ubuntu-latest is currently being updated to point to ubuntu-22.04, which uses glibc 2.35. Since your binary is not compiled statically, this means for the next doc2dash release you won't support Ubuntu 20.04 and other systems with an older glibc.

    And the current release does not support Ubuntu 18.04, as shown by pyoxidizer analyze:

    $ pyoxidizer analyze doc2dash
    
    ...
    
    glibc
    -----
    
    Minimum Version: 2.29
    Minimum Distro Versions:
      Debian 11
      Fedora 30
      OpenSUSE 15.3
      RHEL 9
      Ubuntu 19.04
    

    See more details in the documentation: https://pyoxidizer.readthedocs.io/en/stable/pyoxidizer_distributing_linux.html#managing-binary-portability-on-linux

    This might not be a problem, but I thought you would like to know.

    opened by pquentin 0
  • automatically import nbviewer

    automatically import nbviewer

    It has become common for python docs to reference jupyter notebooks with external links to http://nbviewer.ipython.org./. (e.g. http://nbviewer.jupyter.org/github/cvxgrp/cvx_short_course/blob/master/applications/portfolio_optimization.ipynb) I have found that these notebooks are often the most useful portion of the docs, however they are not automatically captured by doc2dash. I propose a feature that automatically downloads these notebooks.

    opened by bendichter 1
Releases(3.0.0)
  • 3.0.0(Sep 13, 2022)

    Presenting the biggest doc2dash release ever!

    Most importantly, as you can see below, there's stand-alone binaries for Linux, macOS, and Windows for amd64 platforms! No more Python (if you don't want to)! I found this so exciting, I wrote a short Today-I-Learned post about it. But even if you keep using it via Python, the dependency tree got a lot slimmer since it doesn't depend on Sphinx anymore.

    Feature-wise this release introduces first-class support for MkDocs and improves its pydoctor support while at it.

    There's a lot more that happened, including prettier (and more informative) progress bars and smarter handling of imperfect indixes.

    All-in-all, it comes close to a complete rewrite. I hope you enjoy it, and if you haven't played with Dash.app or Zeal until now – now is the perfect moment!

    P.S. Sorry, I also broke your custom plugins. But there's docs on writing your own now!

    Full Changelog

    Removed

    • Since pydoctor added support for intersphinx in version 21.2.0, the dedicated HTML-parsing parser has been removed. If you need to parse legacy pydoctor docs, please use doc2dash 2.4.1.

    Added

    • Stand-alone binaries! If your platform is supported, you can now download binaries from the release page without dealing with Python at all – courtesy of PyOxidizer.
    • intersphinx: documentation based on MkDocs with mkdocstrings metadata is now supported.
    • intersphinx: if no explicit name is passed, the docset name is derived von the documentation's metadata (and not the directory name, which is more often than not just html).
    • We use rich for output and progress bars now. This allows us more granular progress indication.
    • Documentation on writing your own parser plugins.

    Changed

    • Tons of refactorings that probably broke your custom parsers. Sorry about that and let us know, if we can help you fixing them.
    • intersphinx: We now parse objects.inv files on our own. Sphinx is not a dependency anymore.
    • intersphinx: Files that are indexed by intersphinx, but don't exist aren't added to the docset anymore. Common example is py-modindex.html. #113 #115
    • We now check if the index page passed via --index-page / -I exists and fail if it doesn't.
    Source code(tar.gz)
    Source code(zip)
    doc2dash.x86_64-apple-darwin.zip(21.99 MB)
    doc2dash.x86_64-pc-windows-msvc.zip(18.14 MB)
    doc2dash.x86_64-unknown-linux-gnu.zip(33.38 MB)
  • 2.4.1(Jan 21, 2022)

  • 2.4.0(Nov 16, 2021)

    This version dropped every Python version older than 3.8. Please use 2.3 if you need to run doc2dash on legacy Python versions.

    • Better display names with Sphinx v2 inventories. #101
    • Added new intersphinx types:
      • cmdoption (deprecated alias for option)
      • doc
      • label
      • property
      • protocol
      • setting
      • term Re-indexing your documentation may add new index entries!
    • zope.interface is not a dependency anymore.
    • colorama is only a dependency on Windows now.
    • Work around a display bug in older Dash.app releases by setting the plist key DashDocSetDeclaredInStyle to originalName.
    Source code(tar.gz)
    Source code(zip)
  • 2.2.0(Jun 23, 2018)

  • 2.1.0(Dec 1, 2016)

    • Remove legacy Sphinx parsing. The intersphinx parser is much more robust so the old way shouldn’t be needed anymore.
    • Add support for InterSphinx constants. #49
    • Fix handling in-docs links with multiple anchors. #47
    • Add support for --enable-js and --online-redirect-url options. #43
    • Better support for relative paths. #37, #41
    Source code(tar.gz)
    Source code(zip)
  • 2.0.2(Dec 1, 2016)

  • 2.0.1(Dec 1, 2016)

  • 2.0.0(Aug 14, 2014)

    2.0.0 (2014-08-14)

    • Added a new parser for Sphinx documentation that uses intersphinx files that are machine readable. That should lead to more reliable parsing and a better deduction of symbol types. (#28)
    • Added Sphinx-based documentation.
    • Added colors, styles, and a progress bar to make output more comprehensible.
    • setup.py test works now.
    • Internally quite a few changes happened. Most prominently tuples and namedtuples have been replaced by real classes and parsers don't inherit from anything anymore. The fundamental working principal stayed the same though so porting your parsers is trivial.
    Source code(tar.gz)
    Source code(zip)
script to calculate total GPA out of 4, based on input gpa.csv

gpa_calculator script to calculate total GPA out of 4 based on input gpa.csv to use, create a total.csv file containing only one integer showing the t

Mohamad Bastin 1 Feb 07, 2022
💡 Catatan Materi Bahasa Pemrogramman Python

Repository catatan kuliah Andika Tulus Pangestu selama belajar Dasar Pemrograman dengan Python.

0 Oct 10, 2021
A plugin to introduce a generic API for Decompiler support in GEF

decomp2gef A plugin to introduce a generic API for Decompiler support in GEF. Like GEF, the plugin is battery-included and requires no external depend

Zion 379 Jan 08, 2023
Generate YARA rules for OOXML documents using ZIP local header metadata.

apooxml Generate YARA rules for OOXML documents using ZIP local header metadata. To learn more about this tool and the methodology behind it, check ou

MANDIANT 34 Jan 26, 2022
A Material Design theme for MkDocs

A Material Design theme for MkDocs Create a branded static site from a set of Markdown files to host the documentation of your Open Source or commerci

Martin Donath 12.3k Jan 04, 2023
A web app builds using streamlit API with python backend to analyze and pick insides from multiple data formats.

Data-Analysis-Web-App Data Analysis Web App can analysis data in multiple formates(csv, txt, xls, xlsx, ods, odt) and gives shows you the analysis in

Kumar Saksham 19 Dec 09, 2022
A course-planning, course-map rendering and GPA-calculation web service, designed for the SFU (Simon Fraser University) student.

SFU Course Planner What is the overall goal of the project (i.e. what does it do, or what problem is it solving)? As the title suggests, this project

Ash Peng 1 Oct 21, 2021
Loudchecker - Python script to check files for earrape

loudchecker python script to check files for earrape automatically installs depe

1 Jan 22, 2022
Course materials for: Geospatial Data Science

Course materials for: Geospatial Data Science These course materials cover the lectures for the course held for the first time in spring 2022 at IT Un

Michael Szell 266 Jan 02, 2023
Data Inspector is an open-source python library that brings 15++ types of different functions to make EDA, data cleaning easier.

Data Inspector Data Inspector is an open-source python library that brings 15 types of different functions to make EDA, data cleaning easier. Author:

Kazi Amit Hasan 38 Nov 24, 2022
A tutorial for people to run synthetic data replica's from source healthcare datasets

Synthetic-Data-Replica-for-Healthcare Description What is this? A tailored hands-on tutorial showing how to use Python to create synthetic data replic

11 Mar 22, 2022
Markdown documentation generator from Google docstrings

mkgendocs A Python package for automatically generating documentation pages in markdown for Python source files by parsing Google style docstring. The

Davide Nunes 44 Dec 18, 2022
Example Python code for running the mango-explorer marketmaker

🥭 Mango Explorer 📖 Introduction This guide will show you how to load and run a customisable marketmaker that runs on Mango Markets using the mango-e

Blockworks Foundation 2 Apr 11, 2022
A Collection of Cheatsheets, Books, Questions, and Portfolio For DS/ML Interview Prep

Here are the sections: Data Science Cheatsheets Data Science EBooks Data Science Question Bank Data Science Case Studies Data Science Portfolio Data J

James Le 2.5k Jan 02, 2023
Count the number of lines of code in a directory, minus the irrelevant stuff

countloc Simple library to count the lines of code in a directory (excluding stuff like node_modules) Simply just run: countloc node_modules args to

Anish 4 Feb 14, 2022
The OpenAPI Specification Repository

The OpenAPI Specification The OpenAPI Specification is a community-driven open specification within the OpenAPI Initiative, a Linux Foundation Collabo

OpenAPI Initiative 25.5k Dec 29, 2022
MkDocs Plugin allowing your visitors to *File > Print > Save as PDF* the entire site.

mkdocs-print-site-plugin MkDocs plugin that adds a page to your site combining all pages, allowing your site visitors to File Print Save as PDF th

Tim Vink 67 Jan 04, 2023
python wrapper for simple-icons

simpleicons Use a wide-range of icons derived from the simple-icons repo in python. Go to their website for a full list of icons. The slug version mus

Sachin Raja 14 Nov 07, 2022
An open-source script written in python just for fun

Owersite Owersite is an open-source script written in python just for fun. It do

大きなペニスを持つ少年 7 Sep 21, 2022
Quilt is a self-organizing data hub for S3

Quilt is a self-organizing data hub Python Quick start, tutorials If you have Python and an S3 bucket, you're ready to create versioned datasets with

Quilt Data 1.2k Dec 30, 2022