I'm having issues with the ghapi 0.1.17 version released on May, 24th.

I have a private project where I use ghapi to automatically label PRs and issues based on the path of the files mentioned or modified, and the title of the PR/issue. I am using the following public project to test the workflows and scripts: https://github.com/jhlegarreta/labelertest

The labeler on the PR/issue title was working fine with ghapi 0.1.16, but it is systematically returning an HTTP Error 422: Unprocessable Entity error for the following call:

api.issues.add_labels(
            accept="application/vnd.github.v3+json",
            issue_number=pull_request.number, labels=title_labels_to_add)

where the issue_number and labels have the appropriate values.

Here is an example of a successful call using 0.1.16: https://github.com/jhlegarreta/labelertest/runs/2750092407?check_suite_focus=true#step:5:495

And here is an example of a successful call using 0.1.17: https://github.com/jhlegarreta/labelertest/runs/2750038846?check_suite_focus=true#step:5:505

for the same contents from the viewpoint of the files added in the PRs and their titles.

I have not found any release notes notifying about changes, or an older version of the documentation. According to the latest documentation: https://docs.github.com/en/rest/reference/issues#add-labels-to-an-issue

I would be providing the expected parameters to the call.

For what is worth, the 0.1.16 was also working when I only had:

api.issues.add_labels(pull_request.number, title_labels_to_add)

but 0.1.17 failed as described, so the parameters seem not to be the problem.

The relevant GitHub workflow file is https://github.com/jhlegarreta/labelertest/blob/main/.github/workflows/pr_title_labeler.yml, and the Python script containing the calls to the ghapi is https://github.com/jhlegarreta/labelertest/blob/main/.github/scripts/pr_title_regex_labeler.py

Would you please be able to tell what I am doing wrong.

Thanks for providing us with this valuable tool and thanks for the support.

What I'm trying to do

I'm trying to use this library to move a GitHub Project card to the top of its list. The context here is that some cards point to issues with a 🔥 priority label, and I'd like to write a short script that:

• Loops through each card in a column
• If that card points to an issue (via content_url), grab the issue's list of labels
• If priority is in one of the labels, then bump the card to the top of the column.

What's not working

Whenever I try to move a card using ghapi, e.g. via:

api.projects.move_card(59656056, 0)

I get this message:

HTTP422UnprocessableEntityError: HTTP Error 422: Unprocessable Entity

If I debug and look at the URL of the Request object, it doesn't seem quite right

req.full_url
> 'https://api.github.com/projects/columns/cards/59656056/moves'

I'm not sure if I'm doing something wrong, or if this is a bug. So, opening it in case others have a suggestion :-)

The docs don't mention where to get paged. I'm guessing it's from ghapi.page import paged. Anyway, when trying to follow the example I get:

In [82]: repos = api.repos.list_for_org('<...>')                                                                                                                                             
In [83]: len(repos)                                                                                                                                                                                
Out[83]: 30                                                                                                                                                                                        

In [84]: repos_paged = paged(api.repos.list_for_org('<...>'))                                                                                                                                
In [85]: for page in repos_paged: print(len(page), page[0].name)                                                                                                                                   
---------------------------------------------------------------------------                                                                                                                        
TypeError                                 Traceback (most recent call last)                                                                                                                        
<ipython-input-85-f32e698c6d18> in <module>                                                                                                                                                        
----> 1 for page in repos_paged: print(len(page), page[0].name)                                                                                                                                    
                                                                                                                                                                                                   
~/git/github-analysis/lib/python3.8/site-packages/ghapi/page.py in paged(oper, per_page, max_pages, *args, **kwargs)                                                                               
     14 def paged(oper, *args, per_page=30, max_pages=9999, **kwargs):                                                                                                                             
     15     "Convert operation `oper(*args,**kwargs)` into an iterator"                                                                                                                            
---> 16     yield from itertools.takewhile(noop, (oper(*args, per_page=per_page, page=i, **kwargs) for i in range(1,max_pages+1)))                                                                 
     17                                                                                                                                                                                            
     18 # Cell                                                                                                                                                                                     
                                                                                                                                                                                                   
~/git/github-analysis/lib/python3.8/site-packages/ghapi/page.py in <genexpr>(.0)                                                                                                                   
     14 def paged(oper, *args, per_page=30, max_pages=9999, **kwargs):                                                                                                                             
     15     "Convert operation `oper(*args,**kwargs)` into an iterator"                                                                                                                            
---> 16     yield from itertools.takewhile(noop, (oper(*args, per_page=per_page, page=i, **kwargs) for i in range(1,max_pages+1)))                                                                 
     17                                                                                                                                                                                            
     18 # Cell                                                                                                                                                                                     
                                                                                                                                                                                                   
TypeError: 'L' object is not callable                  

Currently the library raises an HTTP 400 Bad Request when scopes are passed to GhDeviceAuth because we attempt to serialise a one-tuple containing an iterable, instead of the iterable itself.

This change fixes auth to maintain the iterable all the way through so that serialisation works and the API can understand the scopes we send it.

Note to reviewers: I've not seen a codebase like this before, it looks like all the code is auto-generated from the notebooks? Can I rely on the package codegen working for me here?

Hello. This is more a question about how to do this... I am trying to create a small Action that would post a comment on a PR after it has been created, but I have no success if the PR comes from a fork. I am using the "getting started" action from the documentation. I get the following error:

>> Creating comment on PR #225
[532](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:532)
Traceback (most recent call last):
[533](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:533)
  File ".github/scripts/open_in_colab.py", line 57, in <module>
[534](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:534)
    create_comment()
[535](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:535)
  File ".github/scripts/open_in_colab.py", line 55, in create_comment
[536](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:536)
    api.issues.create_comment(issue_number=issue, body=body)
[537](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:537)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/ghapi/core.py", line 63, in __call__
[538](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:538)
    return self.client(self.path, self.verb, headers=headers, route=route_p, query=query_p, data=data_p)
[539](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:539)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/ghapi/core.py", line 108, in __call__
[540](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:540)
    res,self.recv_hdrs = urlsend(path, verb, headers=headers or None, debug=self.debug, return_headers=True,
[541](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:541)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/fastcore/net.py", line 212, in urlsend
[542](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:542)
    return urlread(req, return_json=return_json, return_headers=return_headers)
[543](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:543)
  File "/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/fastcore/net.py", line 113, in urlread
[544](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:544)
    if 400 <= e.code < 500: raise ExceptionsHTTP[e.code](e.url, e.hdrs, e.fp) from None
[545](https://github.com/wandb/examples/runs/5813246392?check_suite_focus=true#step:4:545)
fastcore.basics.HTTP403ForbiddenError: HTTP Error 403: Forbidden

Do I need to change something in the admin of the repo?

Adds jwt_token attribute to GhApi to allow users to call app endpoints that require a jwt token instead of a pat/access_key

closes #85

This should fix and allow you to authenticate to the following endpoints:

• apps.get_authenticated(): Get the authenticated app
• apps.list_installations(per_page, page, since, outdated): List installations for the authenticated app
• apps.get_installation(installation_id): Get an installation for the authenticated app
• apps.delete_installation(installation_id): Delete an installation for the authenticated app
• apps.create_installation_access_token(installation_id, repositories, repository_ids, permissions): Create an installation access token for an app
• apps.suspend_installation(installation_id): Suspend an app installation
• apps.unsuspend_installation(installation_id): Unsuspend an app installation
• apps.get_org_installation(org): Get an organization installation for the authenticated app
• apps.get_repo_installation(owner, repo): Get a repository installation for the authenticated app
• apps.get_user_installation(username): Get a user installation for the authenticated app
• apps.get_webhook_config_for_app(): Get a webhook configuration for an app
• apps.update_webhook_config_for_app(url, content_type, secret, insecure_ssl): Update a webhook configuration for an app
• apps.get_subscription_plan_for_account_stubbed(account_id): Get a subscription plan for an account (stubbed)
• apps.list_plans_stubbed(per_page, page): List plans (stubbed)
• apps.list_accounts_for_plan_stubbed(plan_id, sort, direction, per_page, page): List accounts for a plan (stubbed)

And probably more, these were just the ones I quickly tested.

For example:

jwt = create_jwt(app_id, private_key)

app_api = GhApi(jwt_token=jwt)

print("List Installations")
installations = app_api.apps.list_installations()
print(installations)

print("Get Access Token")
id = installations[0]['id']
print(app_api.apps.create_installation_access_token(id))

List Installations

[- id: 1111
- account: 
  - login: ffalor
  - id: 1
  - node_id: MDQ6VXNlcjM1MTQ0MTQx
  #etc...]

Get Access Token
- token: ghs_324324234234
- expires_at: 2021-08-21T11:40:36Z
- permissions: 
  - administration: read
  - contents: read
  - issues: write
  - metadata: read
- repository_selection: all

Not 100% what all I can update since from my understanding this project is auto generated. I did not see any tests to update. Please let me know what would need to change in order to add the ability to provide a jwt_token.

Reopen of #87 had to fix line ending changes and auto formatting.

For some endpoints GitHub lets you request different response format using the Accept header [1]. However, by default if the response is not in JSON format GhApi.__call__ will raise an error.

This commit makes it possible by adding a bit of logic to look at the Accept header in the request and tell fastcore.core.urlsend not to return JSON if it doesn't look like the user is requesting a JSON media type.

I just found a gotcha where I was trying to update a github label that had a space in it. Trying to do so was raising a 400 error. I fixed it by doing:

from urllib.parse import quote
quote(label["name"])

It would be helpful to either automate this under-the-hood, or to raise a more helpful error that suggests this might be the issue.

I might obviously be missing something, but I'm getting a UnicodeDecodeError, when trying to download an artifact:

>>> api.actions.download_artifact("YannickJadoul", "Parselmouth", 28315202, "zip")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/yannick/.local/lib/python3.6/site-packages/ghapi/core.py", line 60, in __call__
    return self.client(self.path, self.verb, headers=headers, route=route_p, query=query_p, data=data_p)
  File "/home/yannick/.local/lib/python3.6/site-packages/ghapi/core.py", line 104, in __call__
    route=route or None, query=query or None, data=data or None)
  File "/home/yannick/.local/lib/python3.6/site-packages/fastcore/net.py", line 175, in urlsend
    return urlread(req, return_json=return_json, return_headers=return_headers)
  File "/home/yannick/.local/lib/python3.6/site-packages/fastcore/net.py", line 115, in urlread
    if decode: res = res.decode()
UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 68-69: invalid continuation byte

It seems like the download bytes are attempted to be decoded to a unicode string? I hope I didn't miss any advice in the docs on how to disable this.

The issue doesn't seem to be happening for e.g. git.get_blob, because the response there is base64-encoded, while GhApi.get_content does return some bytes (though it's not possible to download artifacts this way, as far as I know).

This allows users to create multiple instances of GhApi, eg to communicate to both github.com and a private GitHub Enterprise instance.

gh = GhApi()
ghe = GhApi(gh_host="https://github.example.com")

It also allows scripts to define a custom GitHub Enterprise host to be used, without requiring the user to remember to pass it explicitly in the environment.

Description of the problem

Hello Jeremy and Hamel,

I'm trying to use ghapi to fetch issues from a GitHub repo and then dump them to disk as a jsonl file, where each line is an issue.

The problem I'm running into is that the result from GhAPI.issues.list_for_repo contains fastcore's special L type which appears to not be JSON serialisable. As a result I'm getting the following error: TypeError: Object of type L is not JSON serializable

Is there a way to make L play nice with JSON?

Steps to reproduce

import json
from ghapi.all import GhApi

api = GhApi()
issues = api.issues.list_for_repo("huggingface", "transformers", page=1, per_page=5, state="all")

# try to write the first issue
with open("issues.jsonl", "w") as f:
    json.dump(issues[0], f)

Expected result

I can write GitHub issues to disk in JSON format.

Actual result

Here's the stack trace

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-48-04139a7641e4> in <module>
      6 
      7 with open("issues.json", "w") as f:
----> 8     json.dump(issues[0], f)

~/miniconda3/envs/transformersbook/lib/python3.8/json/__init__.py in dump(obj, fp, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, default, sort_keys, **kw)
    177     # could accelerate with writelines in some versions of Python, at
    178     # a debuggability cost
--> 179     for chunk in iterable:
    180         fp.write(chunk)
    181 

~/miniconda3/envs/transformersbook/lib/python3.8/json/encoder.py in _iterencode(o, _current_indent_level)
    429             yield from _iterencode_list(o, _current_indent_level)
    430         elif isinstance(o, dict):
--> 431             yield from _iterencode_dict(o, _current_indent_level)
    432         else:
    433             if markers is not None:

~/miniconda3/envs/transformersbook/lib/python3.8/json/encoder.py in _iterencode_dict(dct, _current_indent_level)
    403                 else:
    404                     chunks = _iterencode(value, _current_indent_level)
--> 405                 yield from chunks
    406         if newline_indent is not None:
    407             _current_indent_level -= 1

~/miniconda3/envs/transformersbook/lib/python3.8/json/encoder.py in _iterencode(o, _current_indent_level)
    436                     raise ValueError("Circular reference detected")
    437                 markers[markerid] = o
--> 438             o = _default(o)
    439             yield from _iterencode(o, _current_indent_level)
    440             if markers is not None:

~/miniconda3/envs/transformersbook/lib/python3.8/json/encoder.py in default(self, o)
    177 
    178         """
--> 179         raise TypeError(f'Object of type {o.__class__.__name__} '
    180                         f'is not JSON serializable')
    181 

TypeError: Object of type L is not JSON serializable

Versions

• ghapi: v0.1.16

pages function returns only the first page of packages

api = GhApi(token=gh_token)
pack_pages = pages(api.packages.list_packages_for_organization, 5, org="my_org", package_type="npm")
for page in pack_pages:
    for item in page:
        print(item['name'])

this code will print name of the packages from the first page only

I'm seeing some weirdness in re: ProjectsV2. Specifically, this command:

gh api graphql -f query='
  query{
  node(id: "PVT_kwDOAU_qk84AHJ4X") {
    ... on ProjectV2 {
      fields(first: 20) {
        nodes {
          ... on ProjectV2Field {
            id
            name
          }
          ... on ProjectV2IterationField {
            id
            name
            configuration {
              iterations {
                startDate
                id
              }
            }
          }
          ... on ProjectV2SingleSelectField {
            id
            name
            options {
              id
              name
            }
          }
        }
      }
    }
  }
}'

Works fine against this repo (https://github.com/orgs/filecoin-project/projects/65/views/1).

However, the same thing in the API:

gh_token = os.getenv("GITHUB_TOKEN")
api = GhApi(owner="filecoin-project", token=gh_token)

p = api.projects.list_columns(node="PVT_kwDOAU_qk84AHJ4X", per_page=20, page=0)
print(f"{p.title} - {p.id}")

Returns not found. Querying issues normally works. Could this be a v2 issue?

I ran api.repos.create_fork(owner="WebGoat", repo="WebGoat"), and got an error:

Traceback (most recent call last):
  File "/Users/iulspop/Development/github-app/scripts/e2e/main.py", line 15, in <module>
    api.repos.create_fork(owner="WebGoat", repo="WebGoat")
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/ghapi/core.py", line 61, in __call__
    return self.client(self.path, self.verb, headers=headers, route=route_p, query=query_p, data=data_p)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/ghapi/core.py", line 120, in __call__
    res,self.recv_hdrs = urlsend(path, verb, headers=headers or None, debug=debug, return_headers=True,
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fastcore/net.py", line 218, in urlsend
    return urlread(req, return_json=return_json, return_headers=return_headers)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/fastcore/net.py", line 119, in urlread
    if 400 <= e.code < 500: raise ExceptionsHTTP[e.code](e.url, e.hdrs, e.fp, msg=e.msg) from None
fastcore.net.HTTP404NotFoundError: HTTP Error 404: Not Found
====Error Body====
{
  "message": "Not Found",
  "documentation_url": "https://docs.github.com/rest/reference/repos#create-a-fork"
}

But when I run the curl command directly:

curl \
  -X POST \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer X" \
  https://api.github.com/repos/WebGoat/WebGoat/forks

It just works.

Any ideas why the ghapi isn't working here?

afaics the use of fast core types like L prevents basic operations like serializing the response from ghapi.

 >>> json.dumps(o)
*** TypeError: Object of type L is not JSON serializable

Hi, I was having trouble adding an issue as a card on a project using api.projects.create_card. The below stack overflow described my situation exactly, so thanks to the author, and another kind person provided a workaround/solution. I'm also surprised I did not find an issue already opened on this (if I missed it, please forgive me).

https://stackoverflow.com/questions/70306360/github-api-add-an-issue-to-a-project

What I encounter when passing the content_id with the issue id and content_type as 'Issue' is a Validation Failed error message.

api.projects.create_card(column_id=col.id, content_id=issue.id, content_type='Issue')

====Error Body====
{
  "message": "Validation Failed",
  "errors": [
     {
        "resource": "Project Card",
        "code": "unprocessable",
        "field": "data",
        "message": "You must provide either a note or a contentId."
     }
  ],
  "documentation_url": "https://docs.github.com/enterprise/3.3/v3/projects/cards/#create-a-project-card"
}

This works

curl -v \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer $GH_ENTERPRISE_TOKEN" \
  https://$GH_HOST/api/v3/search/issues\?q\=repo:$REPO_NAME+state:open\&per_page=30\&page=

This does not

from ghapi.all import *
api = GhApi(<token etc..>)
paged(api.search.issues_and_pull_requests, q="repo:<REPO_NAME> state:open')

For debug log debug=print_summary

https://<HOST>/api/v3/search/issues?q=repo%3A<OWNER>%2F<REPO>+state%3Aopen&per_page=30&page=1

See : was replaced with %3A. This I think is what breaks the query, some uri/url encoding happening somewhere