Build documentation in multiple repos into one site.

Overview

mkdocs-multirepo-plugin

Build documentation in multiple repos into one site.

Setup

Install plugin using pip:

pip install git+https://github.com/jdoiro3/mkdocs-multirepo-plugin

Next, add the plugin to your mkdocs.yml

plugins:
  - multirepo

The plugin introduces the !import statement in your config's nav section. You can now use the import statement to add a documentation section, where the docs are pulled from the source repo.

nav takes precedence over repos (see below).

nav:
  - Home: 'index.md'
  - MicroService: '!import https://github.com/{user}/{repo name}@{branch}'

If you'd prefer MkDocs to build the site nav based on the directory structure, you can define your other repos within the plugins section.

plugins:
  - multirepo:
      cleanup: True # (optional) tells multirepo to cleanup the temporary directory where other repo docs are imported to
      folder_name: multirepo_docs # (optional) tells multirepo what the temp directory should be called
      repos:
        - section: Backstage
          import_url: 'https://github.com/backstage/backstage'
        - section: Monorepo
          import_url: 'https://github.com/backstage/mkdocs-monorepo-plugin'
        - section: 'Techdocs-cli'
          import_url: 'https://github.com/backstage/[email protected]'

Notes

  • If both repos and nav is specified in mkdocs.yml, repos are ignored.

TODO

  • Add tests (no one will use it if it isn't tested well).
  • Add Linux support
    • Looks like git clone --sparse doesn't work with urls on Linux make sure git version is up to date
  • Figure out how tech writers can develop on local copies of repos and see changes in the site before pushing. Maybe using symbolic links.
Comments
  • Adds the access token for git only when executing a bash script for git

    Adds the access token for git only when executing a bash script for git

    The docs for an external repo are supposed to be included in the main menu directly without a docs sub menu entry. To achieve this, the contents of the docs folder inside a repository is moved up directly to the folder of the repo checkout.

    This feature was broken as the code tried to insert the AccessToken for git into the first argument, even though in this case the first argument is a folder name, not a git URL. This led to the the move failing (silently).

    opened by dhs3000 11
  • Building always fails - No such file or directory

    Building always fails - No such file or directory

    Hi, I'd like to thank you for making this awesome plugin, first and foremost! ♥

    Bug Description

    Every configuration I tried leads to a FileNotFoundError: [Errno 2] No such file or directory: exception. I can't figure out whether it's my fault or a real bug in the plugin.

    Let's assume we have the following, "main" doc site we're trying to build/serve:

    site_name: My Docs
    nav:
      - Home: 'index.md'
      - Foo: '!import {url}?branch=docs/mkdocs-bringup'
    plugins:
      - multirepo
    

    Note: I've intentionally hid the URL, but it's an enterprise Gitlab, if that's of any help. The real name of the project, currently represented as Foo, is also hidden.

    Foo contains a valid mkdocs.yml, with Nav properly defined.

    Now, when calling mkdocs build or mkdocs serve, I get this: FileNotFoundError: [Errno 2] No such file or directory: PosixPath('/home/timorg/my-docs/temp_dir/Foo')
    I also get this when moving the import from the Nav section to the plugin's config, i.e. listing sections.

    Logs

    Here's what I got when running mkdocs build --verbose:

    DEBUG    -  Loading configuration file: /home/timorg/service-development-docs/mkdocs.yml
    DEBUG    -  Loaded theme configuration for 'mkdocs' from '/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/themes/mkdocs/mkdocs_theme.yml': {'static_templates': ['404.html'], 'locale':
                'en', 'include_search_page': False, 'search_index_only': False, 'highlightjs': True, 'hljs_languages': [], 'hljs_style': 'github', 'navigation_depth': 2, 'nav_style': 'primary', 'analytics':
                {'gtag': None}, 'shortcuts': {'help': 191, 'next': 78, 'previous': 80, 'search': 83}}
    DEBUG    -  Config value: 'config_file_path' = '/home/timorg/my-docs/mkdocs.yml'
    DEBUG    -  Config value: 'site_name' = 'Service Development Docs'
    DEBUG    -  Config value: 'nav' = [{'Home': 'index.md'}, {'Datastore': '!import {url}?branch=docs/mkdocs-bringup'}]
    DEBUG    -  Config value: 'pages' = None
    DEBUG    -  Config value: 'site_url' = ''
    DEBUG    -  Config value: 'site_description' = None
    DEBUG    -  Config value: 'site_author' = None
    DEBUG    -  Config value: 'theme' = Theme(name='mkdocs', dirs=['/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/themes/mkdocs',
                '/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/templates'], static_templates=['404.html', 'sitemap.xml'], locale=Locale(language='en', territory=''),
                include_search_page=False, search_index_only=False, highlightjs=True, hljs_languages=[], hljs_style='github', navigation_depth=2, nav_style='primary', analytics={'gtag': None}, shortcuts={'help':
                191, 'next': 78, 'previous': 80, 'search': 83})
    DEBUG    -  Config value: 'docs_dir' = '/home/timorg/service-development-docs/docs'
    DEBUG    -  Config value: 'site_dir' = '/home/timorg/service-development-docs/site'
    DEBUG    -  Config value: 'copyright' = None
    DEBUG    -  Config value: 'google_analytics' = None
    DEBUG    -  Config value: 'dev_addr' = Address(host='127.0.0.1', port=8000)
    DEBUG    -  Config value: 'use_directory_urls' = True
    DEBUG    -  Config value: 'repo_url' = ''
    DEBUG    -  Config value: 'repo_name' = ''
    DEBUG    -  Config value: 'edit_uri' = ''
    DEBUG    -  Config value: 'extra_css' = []
    DEBUG    -  Config value: 'extra_javascript' = []
    DEBUG    -  Config value: 'extra_templates' = []
    DEBUG    -  Config value: 'markdown_extensions' = ['toc', 'tables', 'fenced_code']
    DEBUG    -  Config value: 'mdx_configs' = {}
    DEBUG    -  Config value: 'strict' = False
    DEBUG    -  Config value: 'remote_branch' = 'gh-pages'
    DEBUG    -  Config value: 'remote_name' = 'origin'
    DEBUG    -  Config value: 'extra' = {}
    DEBUG    -  Config value: 'plugins' = PluginCollection([('multirepo', <mkdocs_multirepo_plugin.plugin.MultirepoPlugin object at 0x7f3746f10a90>)])
    DEBUG    -  Config value: 'watch' = None
    INFO     -  Multirepo plugin importing docs...
                        :   0%|                                                                                                                                                                | 0/1 [00:00<?, ?it/s]Traceback (most recent call last):
      File "/home/timorg/.local/bin/mkdocs", line 8, in <module>
        sys.exit(cli())
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
        return self.main(*args, **kwargs)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1055, in main
        rv = self.invoke(ctx)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/click/core.py", line 760, in invoke
        return __callback(*args, **kwargs)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/__main__.py", line 192, in build_command
        build.build(config.load_config(**kwargs), dirty=not clean)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/commands/build.py", line 257, in build
        config = config['plugins'].run_event('config', config)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs/plugins.py", line 102, in run_event
        result = method(item, **kwargs)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/plugin.py", line 150, in on_config
        return self.handle_nav_based_import(config)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/plugin.py", line 103, in handle_nav_based_import
        asyncio_run(batch_import(repos))
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/util.py", line 115, in asyncio_run
        asyncio.run(futures)
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
        return future.result()
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/structure.py", line 255, in batch_import
        repo = await import_async
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/tasks.py", line 611, in _wait_for_one
        return f.result()  # May raise f.exception().
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/structure.py", line 237, in import_docs
        await execute_bash_script("mv_docs_up.sh", [self.docs_dir.replace("/*", "")], cwd=self.location)
      File "/home/timorg/.local/pipx/venvs/mkdocs/lib/python3.9/site-packages/mkdocs_multirepo_plugin/util.py", line 97, in execute_bash_script
        process = await asyncio.create_subprocess_exec(
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/subprocess.py", line 236, in create_subprocess_exec
        transport, protocol = await loop.subprocess_exec(
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/base_events.py", line 1676, in subprocess_exec
        transport = await self._make_subprocess_transport(
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/unix_events.py", line 197, in _make_subprocess_transport
        transp = _UnixSubprocessTransport(self, protocol, args, shell,
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/base_subprocess.py", line 36, in __init__
        self._start(args=args, shell=shell, stdin=stdin, stdout=stdout,
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/asyncio/unix_events.py", line 789, in _start
        self._proc = subprocess.Popen(
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/subprocess.py", line 951, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/home/timorg/.pyenv/versions/3.9.12/lib/python3.9/subprocess.py", line 1821, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    FileNotFoundError: [Errno 2] No such file or directory: PosixPath('/home/timorg/my-docs/temp_dir/Foo')
                        :   0%|
    

    Note that I've yet again replaced namings in the log to match those defined above. Sorry but it's company-internal info.

    Environment

    OS: Linux (WSL on top of Windows 10) Distro: Ubuntu 20.04 Python: 3.9.12 (Built with pyenv)

    opened by MrPointer 11
  • Issues when running multirepo

    Issues when running multirepo

    When i run mkdocs serve I get the following output.

    INFO     -  Building documentation...
    WARNING  -  Multirepo plugin is ignoring plugins.multirepo.repos. Nav takes precedence
    INFO     -  Multirepo plugin importing docs...
    INFO     -  Cleaning site directory
    INFO     -  Multirepo plugin is cleaning up temp_dir/
    INFO     -  Documentation built in 0.23 seconds
    INFO     -  [16:55:53] Watching paths for changes: 'docs', 'mkdocs.yml'
    INFO     -  [16:55:53] Serving on http://127.0.0.1:8000/
    WARNING  -  [16:55:54] "GET /quix-apis/ HTTP/1.1" code 404
    INFO     -  [16:56:07] Browser connected: http://localhost:8000/
    

    This is my yaml file.. I mostly copied the bits from your readme.

    site_name: My Docs
     
    nav:
      - Home: 'index.md'
      
    plugins:
      - search
      - multirepo:
          # (optional) tells multirepo to cleanup the temporary directory after site is built.
          cleanup: true
          repos:
            - section: Backstage
              # you can define the edit uri path
              import_url: 'https://github.com/backstage/backstage?edit_uri=/blob/master/'
            - section: Monorepo
              import_url: 'https://github.com/backstage/mkdocs-monorepo-plugin?edit_uri=/blob/master/'
            - section: 'Techdocs-cli'
              # note that the branch is still specified in the url
              import_url: 'https://github.com/backstage/techdocs-cli?branch=main&edit_uri=/blob/main/'
            - section: FastAPI
              import_url: 'https://github.com/tiangolo/fastapi?docs_dir=docs/en/docs/*'
            - section: Monorepo Multi Docs
              import_url: https://github.com/backstage/mkdocs-monorepo-plugin?multi_docs=True&docs_dir=sample-docs/*
            - section: 'Django REST'
              section_path: python # Put this under the python menu entry
              import_url: 'https://github.com/encode/django-rest-framework'
            - section: 'Cookiecutter Pypackage'
              section_path: python # Put this under the python menu entry
              import_url: 'https://github.com/zillionare/cookiecutter-pypackage'
            - section: 'Pydantic'
              section_path: python # Put this under the python menu entry
              import_url: 'https://github.com/samuelcolvin/pydantic?branch=main'
    

    I wasn't sure from the docs whether I needed the nav section as well as the plugins config. But without the nav section it fails to build. I noticed WITH the nav section that the output is saying Nav take precedence.

    opened by SteveRosam 10
  • Clarity in the documentation

    Clarity in the documentation

    Hello,

    To an end user of this plugin, it is not clear if the plugin even builds the documentation in the other repositories that it references. It seems to me that the plugin does the following: 1)Reference already built documentation in other repositories and make use of them in this parent repository.

    The plugin does not really build the documentation (i.e run mkdocs build ) in each of the other repositories that the main repository references. This should be made crystal clear so that end users know what the plugin is capable of.

    The current wording in the documentation is ambiguous and is not clear.`

    Thanks, Kiran Hegde

    documentation 
    opened by kirannhegde 9
  • deep nav imports feature

    deep nav imports feature

    needed this for my own personal use - hopefully it will seem like a logical feature to you.

    i ran the tests in my fork and everything was passing, excluding the publish (since i dont have the pypi secret)

    @jdoiro3

    opened by reedsutliff 8
  • nav links to /!import%20url resulting in 404 warning

    nav links to /!import%20url resulting in 404 warning

    Thanks for the plugin. I am trying to set it up with the following parent mkdocs.yml:

    site_name: Documentation
    
    use_directory_urls: false # Either true or false results in the same issue
    
    plugins:
      - search
      - multirepo
    
    nav:
      - Main: index.md
      - DemoRepo: '!import https://github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main'
    

    This builds/serves fine and I can see a temporary folder being created where demo repo is cloned. However, when I try to navigate to the "DemoRepo" menu item I get a 404 page because the link is:

    http://127.0.0.1:8000/!import%20https:/github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main

    I also see a warning:

    WARNING - [15:35:39] "GET /!import%20https:/github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main HTTP/1.1" code 404

    The python version I am using is 3.9.10 on Windows and the following mkdocs versions:

    mkdocs                     1.2.3
    mkdocs-material            8.1.8
    mkdocs-material-extensions 1.0.3
    mkdocs-multirepo-plugin    0.2.6
    

    Do you have any idea what might cause this issue?

    bug additional testing needed 
    opened by fjp 8
  • Adds specific GitHub access token

    Adds specific GitHub access token

    We are using a GitHub App to create an access token rather than using a personal access token (PAT). Using this token with the environment variable AccessToken yields the git error message fatal: could not read Username for 'https://github.com/': No such device or address.

    This PR introduces a new GithubAccessToken environment variable, which is used in the git URL as https://x-access-token:[email protected]/.... without setting the extraheader authorization header (GitHub docs). This approach works with a PAT and an access token from a GitHub app (I tested the former locally from inside a docker container to prevent config leaking).

    The functionality for the existing AccessToken environment variable stays as it is, as it might be different for other environments/git servers.


    • This addresses the issues #13 and initial description of #33.
    • This replaces #54 as the access token is now inserted into the URL inside the bash script and not in util.execute_bash_script which is also used for moving the docs folder.
    opened by dhs3000 7
  • Sparse checkout not working with recent git versions

    Sparse checkout not working with recent git versions

    GitHub Actions recently updated their virtual environments (see ubuntu-latest for example) and now git cli version is 2.37. This is causing problems with git sparse-checkout set used in sparse_clone.sh script because it doesn't admit patterns to be passed.

    Looking at the git docs for a version that works and the one that doesn't seems that we we need to pass --no-cone flag in order to keep using patterns in the sparse checkout. I tested locally modifying the script in my local virtual environment and works but I'm not sure if it's the "real" fix, not very familiar with sparse checkouts.

    opened by rmunoz 7
  • ValueError: max() arg is an empty sequence

    ValueError: max() arg is an empty sequence

    while trying out the plugin, I enabled it after the installation with:

    plugins:
      - multirepo
    

    After that I wanted to start the builtin development server with mkdocs serve. But while doing so I get the following error:

    Error Message

    INFO     -  Building documentation...
    INFO     -  Multirepo plugin importing docs...
    Traceback (most recent call last):
      File "/home/vscode/.local/bin/mkdocs", line 8, in <module>
        sys.exit(cli())
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
        return self.main(*args, **kwargs)
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1053, in main
        rv = self.invoke(ctx)
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
        return _process_result(sub_ctx.command.invoke(sub_ctx))
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
        return ctx.invoke(self.callback, **ctx.params)
      File "/home/vscode/.local/lib/python3.10/site-packages/click/core.py", line 754, in invoke
        return __callback(*args, **kwargs)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/__main__.py", line 177, in serve_command
        serve.serve(dev_addr=dev_addr, livereload=livereload, **kwargs)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/commands/serve.py", line 54, in serve
        config = builder()
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/commands/serve.py", line 49, in builder
        build(config, live_server=live_server, dirty=dirty)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/commands/build.py", line 257, in build
        config = config['plugins'].run_event('config', config)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs/plugins.py", line 102, in run_event
        result = method(item, **kwargs)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs_multirepo_plugin/plugin.py", line 165, in on_config
        return self.handle_nav_based_import(config)
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs_multirepo_plugin/plugin.py", line 117, in handle_nav_based_import
        asyncio_run(batch_import(docs_repo_objs))
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs_multirepo_plugin/util.py", line 117, in asyncio_run
        asyncio.run(futures)
      File "/usr/local/lib/python3.10/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/usr/local/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
        return future.result()
      File "/home/vscode/.local/lib/python3.10/site-packages/mkdocs_multirepo_plugin/structure.py", line 185, in batch_import
        longest_desc = max([len(f"✅ Got {repo.name} Docs") for repo in repos])
    ValueError: max() arg is an empty sequence
    The terminal process "/usr/bin/zsh '-c', 'mkdocs serve'" terminated with exit code: 1.
    

    Am I missing something?

    bug 
    opened by tepene 7
  • Unable to checkout child imports with github actions

    Unable to checkout child imports with github actions

    I am creating a 'documentation' repo in github which hosts a stub mkdocs.yml file under which mkdocs-multirepo-plugin should build documentation from other repos. The .yml is:

    site_name: MyOrg Docs
    theme: readthedocs
    plugins:
      - multirepo
      - search
    
    nav:
      - Home: 'index.md'
      - Included Services:
        - Service1: '!import https://github.com/myOrg/myRepo1?docs_dir=mydocs/*?multi_docs=True?config=mydocs/mkdocs.yml'
        - Service2: '!import https://github.com/myOrg/myRepo2?docs_dir=mydocs/*?multi_docs=True?config=mydocs/mkdocs.yml'
    

    When I run mkdocs buildfrom my machine, it works.

    I want to trigger the build of docs from a github action. Per the documentation, I added github personal access token (PAT) to the repo secrets and included that in my workflow. The workflow .yml is:

    name: Build and Publish Docs 
    
    
    on:
      push:
        branches: ['develop']
    
    env:
      AWS_DEFAULT_REGION: ${{ secrets.AWS_REGION }}
      AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
      AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    
    jobs:
    
      build_and_publish:
        runs-on: ubuntu-latest
    
        steps:
          - name: Checkout
            uses: actions/[email protected]
    
          - name: Install Python
            uses: actions/[email protected]
            with:
              python-version: 3.8
    
          - name: Install pip requirements
            run: |
              python -m pip install --upgrade pip
              pip install -r requirements.txt
    
          - name: Run mkdocs
            run: mkdocs build
            working-directory: docs-root
            env:
              AccessToken: ${{ secrets.MY_PAT }}
    
          #- name: Publish to S3
    

    The output of the 'Run mkdocs' step is:

    Run mkdocs build
      mkdocs build
      shell: /usr/bin/bash -e {0}
      env:
        AWS_DEFAULT_REGION: 
        AWS_SECRET_ACCESS_KEY: 
        AWS_ACCESS_KEY_ID: 
        pythonLocation: /opt/hostedtoolcache/Python/3.8.1[2](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:2)/x64
        LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/[3](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:3).8.12/x6[4](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:4)/lib
        AccessToken: ***
    INFO     -  Multirepo plugin importing docs...
    
                          :   0%|          | 0/2 [00:00<?, ?it/s]Traceback (most recent call last):
      File "/opt/hostedtoolcache/Python/3.8.12/x64/bin/mkdocs", line 8, in <module>
        sys.exit(cli())
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
        return self.main(*args, **kwargs)
      
    <<snip>>
    
        stdout = await execute_bash_script("sparse_clone.sh", args, self.temp_dir)
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/mkdocs_multirepo_plugin/util.py", line 109, in execute_bash_script
        raise GitException(f"\n{stderr}\n")
    mkdocs_multirepo_plugin.util.GitException: 
    Cloning into 'PIN Service'...
    fatal: could not read Username for 'https://github.com': No such device or address
    
    
    Exception ignored in: <function BaseSubprocessTransport.__del__ at 0x7fe74cd8e280>
    Traceback (most recent call last):
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/asyncio/base_subprocess.py", line 1[26](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:26), in __del__
        self.close()
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/asyncio/base_subprocess.py", line 104, in close
        proto.pipe.close()
      
    <<snip>>
    
      File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/asyncio/base_events.py", line [50](https://github.com/BuddyTV/papyrus/runs/6177823965?check_suite_focus=true#step:5:50)8, in _check_closed
    Error:     raise RuntimeError('Event loop is closed')
    RuntimeError: Event loop is closed
    
                          :   0%|          | 0/2 [00:00<?, ?it/s]
    Error: Process completed with exit code 1.
    

    What am I doing wrong?

    Change needed 
    opened by matthewmucker-vizio 6
  • Doesn't work for self hosted github enterprise?

    Doesn't work for self hosted github enterprise?

    I think it doesn't work for self-hosted github enterprise. The URL here is git..com (only vpn accessible), and when running mkdocs serve:

    INFO     -  Multirepo plugin is importing docs for section Section 1
    [Errno 2] No such file or directory: 'where'
    

    This is an example of the mkdocs.yml:

    # Project information
    site_name: Platform Wiki
    site_description: Centralized location for all documentation
    site_author: Platform
    
    # Repository
    repo_name: platform/centralized-docs
    repo_url: https://git.<company>.com/platform/centralized-docs
    
    nav:
      - Home: index.md
      - Login Server: '!import https://git.<company>.com/<org>/<repo>@master'
    
    # Configuration
    theme:
      name: material
      palette:
        primary: indigo
        accent: pink
        scheme: default
    
    plugins:
      - multirepo:
          # (optional) tells multirepo to cleanup the temporary directory after site is built.
          cleanup: true
          # (optional) tells multirepo what the temp directory should be called
          temp_dir: multirepo_docs
    
    
    bug 
    opened by acastro2 5
  • Temp dir issue

    Temp dir issue

    I'm getting this when trying to build.

    The process cannot access the file because it is being used by another process

    I have deleted the temp_dir and still get the same. Has anyone found a work around and is there a build flag or argument to change the temp directory?

    opened by SteveRosam 1
  • Create .git/info dir if it does not exist

    Create .git/info dir if it does not exist

    git init does not necessarily create .git/info, might be version-dependent.

    Fixes #58 (but does not address the general error checking issue of this script, i.e. failing intermediate commands will still get "swallowed").

    opened by hjoukl 2
  • sparse_clone_old.sh might not do a sparse checkout after all

    sparse_clone_old.sh might not do a sparse checkout after all

    git init might not produce a .git/info dir:

    $ mkdir myrepo
    $ cd myrepo/
    $ git init
    Initialized empty Git repository in /tmp/myrepo/.git/
    $ ls -l .git/
    total 8
    -rw-r--r--. 1 holger intern 92 Oct  6 16:19 config
    -rw-r--r--. 1 holger intern 23 Oct  6 16:19 HEAD
    drwxr-xr-x. 2 holger intern 27 Oct  6 16:19 hooks/
    drwxr-xr-x. 4 holger intern 30 Oct  6 16:19 objects/
    drwxr-xr-x. 4 holger intern 31 Oct  6 16:19 refs/
    $ git --version
    git version 2.24.1
    

    Should probably add an mkdir line:

    ...
    git remote add -f origin "$url"                     
    mkdir -p .git/info  # this directory might not exist                           
    for dir in "${dirs[@]}"
    ...
    

    Note that you won't really notice this currently since the shell script will simply continue for the failing commands. This could be mitigated by using set -e errexit or (better) explicit error checking.

    Best, Holger

    opened by hjoukl 2
  • Symlinks from docs dir to toplevel project dir do not work

    Symlinks from docs dir to toplevel project dir do not work

    Hi,

    nice plugin!

    Suppose you have a repo myrepowith toplevel LICENSE and AUTHORS documents that are symlinked from the docs folder, to avoid duplication but use this info in the mkdocs-generated documentation:

    AUTHORS.md
    LICENSE.md
    docs
    ├── about
    │   ├── AUTHORS.md -> ../../AUTHORS.md
    │   └── LICENSE.md -> ../../LICENSE.md
    ├── index.md
    mkdocs.yml
    

    This works with mkdocs serve|build when the repo is used "standalone".

    However, if you want such a repo to be imported with multirepo-plugin:

    # mkdocs.yml of the importing repo
    nav:
      - Home: index.md
      - Documentation:
        - myrepo: '!import https://vcs.example.org/user/myrepo?branch=feature/great-improvements&docs_dir=docs&multi_docs=False&config=mkdocs.yml&extra_imports=["AUTHORS.md", "LICENSE.md"]'
    

    ... the symlinks get broken in the temp_dir sparse checkout:

    $ tree temp_dir/
    temp_dir/
    └── documentation
        └── myrepo
            ├── about
            │   ├── AUTHORS.md -> ../../AUTHORS.md
            │   └── LICENSE.md -> ../../LICENSE.md
            ├── AUTHORS.md
            ├── index.md
            ├── LICENSE.md
            ├── mkdocs.yml
    

    This is due to the way the docs folders contents are put into temp_dir/documentation/myrepo directly, which changes the nesting level. This makes the symlinks invalid.

    Making the links the other way round (e.g. `AUTHORS.md --> docs/about/AUTHORS.md) isn't attractive since you lose the ability to directly render the markdown files in the VCS-GUI (at least in bitbucket, in my case), since the actual git repository content of such symlinks is the path to the symlinked file.

    And you also can't just reference a markdown file in nav: ...outside of the docs_dir.

    Workaround: Duplicate files. Avoid links crossing docs_dir.

    Best regards, Holger

    opened by hjoukl 0
  • including code from paths outside of the mkdocs directory

    including code from paths outside of the mkdocs directory

    Does the plugin support importing code snippets from other paths ?

    We have often things like this in our doc:

    --8<-- "../component/src/test/java/mkdocs/Example.java"
    

    When building a multirepo mkdocs site, it fails:

    ERROR    -  Error reading page 'example/index.md': Snippet at path
                ../component/src/test/java/mkdocs/Example.java could not be found
    

    I am using nav based multirepo import like this:

    Example: '!import [email protected]:myrepo.git?extra_imports=["component/*"]'
    

    The imported repo structure is this:

    component
      src
        test
          java
    mkdocs
      mkdocs.yml
    
    enhancement 
    opened by jorgheymans 3
Releases(v0.4.12)
  • v0.4.12(Oct 21, 2022)

  • v0.4.11(Oct 20, 2022)

    Changes in this Release

    • Added a specific GithubAccessToken environment variable that allows usage of the plugin with GitHub Apps-generated access tokens and personal access tokens.
    • Fixed edit urls for imported repos where the the Mkdocs edit_uri and repo_url aren't set, and the Multirepo edit_uri for an imported repo isn't set. Note that the edit url for imported repos hasn't been robustly tested yet.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.0(Jun 22, 2022)

    Breaking Changes

    • Previously, you would separate config values in an !import statement using multiple ampersands (&). Now, the !import statement will follow general URL syntax, where the url and query string (config values) is separated by one question mark (?) and then each config section is separated by an ampersand. Below is an example !import statement. '!import {url}?branch={branch}&docs_dir={path}&multi_docs={True | False}&config={filename}.yml'

    Other Changes

    • Section names are slugified for clear urls (see PR https://github.com/jdoiro3/mkdocs-multirepo-plugin/pull/28 and Issue https://github.com/jdoiro3/mkdocs-multirepo-plugin/issues/24 for details). This most likely will change in future releases.
    Source code(tar.gz)
    Source code(zip)
  • v0.3.6(May 10, 2022)

  • v0.3.5(Apr 8, 2022)

  • v0.3.3(Apr 1, 2022)

    Additions

    Previously, !import statements could only be included within top level sections in the nav. Now, the !import statement can be located anywhere in the nav.

    nav:
      - Home: index.md
      - microservice:
          - Home: test.md
          - ServiceA: '!import https://github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main'
          - ServiceB: '!import https://github.com/jdoiro3/mkdocs-multirepo-demoRepo1?branch=main'
    
    Source code(tar.gz)
    Source code(zip)
  • v0.3.2(Mar 19, 2022)

    Additions

    • Users can now store the multirepo config file for an imported repo either within the docs directory or the root directory. If the name of the config isn't mkdocs.yml, it can also be specified via the new url parameter: ?config={filename}.yml.

    Changes

    • Fixed the issue where build outputs contained configuration files (see issue #9 for details).
    Source code(tar.gz)
    Source code(zip)
  • v0.3.1(Mar 3, 2022)

    Changes

    • Section names with spaces were causing two directories to be created when the version of git was old (i.e., sparse_clone_old.sh was run).
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Feb 21, 2022)

  • v0.2.9(Feb 19, 2022)

  • v0.2.7(Feb 18, 2022)

  • v0.2.3(Feb 17, 2022)

  • v0.2.0(Feb 10, 2022)

    Alpha Release

    The minimal features of mkdocs-multirepo-plugin have been implemented but have yet to be fully tested.

    Features

    • Build docs from multiple repos into one site
    • Importing repos that are monorepos (not tested well)
    • Run mkdocs serve within an imported repo for local development (not tested well)
    Source code(tar.gz)
    Source code(zip)
Owner
Joseph Doiron
Not an expert yet but I'm trying.
Joseph Doiron
Mkdocs obsidian publish - Publish your obsidian vault through a python script

Mkdocs Obsidian Mkdocs Obsidian is an association between a python script and a

Mara 49 Jan 09, 2023
📖 Generate markdown API documentation from Google-style Python docstring. The lazy alternative to Sphinx.

lazydocs Generate markdown API documentation for Google-style Python docstring. Getting Started • Features • Documentation • Support • Contribution •

Machine Learning Tooling 118 Dec 31, 2022
An awesome Data Science repository to learn and apply for real world problems.

AWESOME DATA SCIENCE An open source Data Science repository to learn and apply towards solving real world problems. This is a shortcut path to start s

Academic.io 20.3k Jan 09, 2023
This is a template (starter kit) for writing Maturity Work with Sphinx / LaTeX at Collège du Sud

sphinx-tm-template Ce dépôt est un template de base utilisable pour écrire ton travail de maturité dans le séminaire d'informatique du Collège du Sud.

6 Dec 22, 2022
pytorch_example

pytorch_examples machine learning site map 정리자료 Resnet https://wolfy.tistory.com/243 convolution 연산 정리 https://gaussian37.github.io/dl-concept-covolut

injae hwang 1 Nov 24, 2021
Watch a Sphinx directory and rebuild the documentation when a change is detected. Also includes a livereload enabled web server.

sphinx-autobuild Rebuild Sphinx documentation on changes, with live-reload in the browser. Installation sphinx-autobuild is available on PyPI. It can

Executable Books 440 Jan 06, 2023
Use Brainf*ck with python!

Brainfudge Run Brainf*ck code with python! Classes Interpreter(array_len): encapsulate all functions into class __init__(self, array_len: int=30000) -

1 Dec 14, 2021
Deduplicating archiver with compression and authenticated encryption.

More screencasts: installation, advanced usage What is BorgBackup? BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports

BorgBackup 9k Jan 09, 2023
A system for Python that generates static type annotations by collecting runtime types

MonkeyType MonkeyType collects runtime types of function arguments and return values, and can automatically generate stub files or even add draft type

Instagram 4.1k Jan 07, 2023
A curated list of awesome tools for Sphinx Python Documentation Generator

Awesome Sphinx (Python Documentation Generator) A curated list of awesome extra libraries, software and resources for Sphinx (Python Documentation Gen

Hyunjun Kim 831 Dec 27, 2022
epub2sphinx is a tool to convert epub files to ReST for Sphinx

epub2sphinx epub2sphinx is a tool to convert epub files to ReST for Sphinx. It uses Pandoc for converting HTML data inside epub files into ReST. It cr

Nihaal 8 Dec 15, 2022
Speed up Sphinx builds by selectively removing toctrees from some pages

Remove toctrees from Sphinx pages Improve your Sphinx build time by selectively removing TocTree objects from pages. This is useful if your documentat

Executable Books 8 Jan 04, 2023
A website for courses of Major Computer Science, NKU

A website for courses of Major Computer Science, NKU

Sakura 0 Oct 06, 2022
This is the repository that includes the code material for the ESweek 2021 for the Education Class Lecture A3 "Learn to Drive (and Race!) Autonomous Vehicles"

ESweek2021_educationclassA3 This is the repository that includes the code material for the ESweek 2021 for the Education Class Lecture A3 "Learn to Dr

F1TENTH Autonomous Racing Community 29 Dec 06, 2022
Toolchain for project structure and documents optimisation

ritocco Toolchain for project structure and documents optimisation

Harvey Wu 1 Jan 12, 2022
The tutorial is a collection of many other resources and my own notes

Why we need CTC? --- looking back on history 1.1. About CRNN 1.2. from Cross Entropy Loss to CTC Loss Details about CTC 2.1. intuition: forward algor

手写AI 7 Sep 19, 2022
Python Advanced --- numpy, decorators, networking

Python Advanced --- numpy, decorators, networking (and more?) Hello everyone 👋 This is the project repo for the "Python Advanced - ..." introductory

Andreas Poehlmann 2 Nov 05, 2021
Markdown documentation generator from Google docstrings

mkgendocs A Python package for automatically generating documentation pages in markdown for Python source files by parsing Google style docstring. The

Davide Nunes 44 Dec 18, 2022
sphinx builder that outputs markdown files.

sphinx-markdown-builder sphinx builder that outputs markdown files Please ★ this repo if you found it useful ★ ★ ★ If you want frontmatter support ple

Clay Risser 144 Jan 06, 2023
level2-data-annotation_cv-level2-cv-15 created by GitHub Classroom

[AI Tech 3기 Level2 P Stage] 글자 검출 대회 팀원 소개 김규리_T3016 박정현_T3094 석진혁_T3109 손정균_T3111 이현진_T3174 임종현_T3182 Overview OCR (Optimal Character Recognition) 기술

6 Jun 10, 2022