RPyC (Remote Python Call) - A transparent and symmetric RPC library for python

Related tags

RPC Serversrpyc
Overview

Version Python Versions Build Status License: MIT

RPyC (pronounced like are-pie-see), or Remote Python Call, is a transparent library for symmetrical remote procedure calls, clustering, and distributed-computing. RPyC makes use of object-proxying, a technique that employs python's dynamic nature, to overcome the physical boundaries between processes and computers, so that remote objects can be manipulated as if they were local.

Documentation can be found at https://rpyc.readthedocs.io

http://rpyc.readthedocs.org/en/latest/_images/screenshot.png

A screenshot of a Windows client connecting to a Linux server.

Note that text written to the server's stdout is actually printed on the server's console.
Comments
  • Remote multithreading stalls client ... and a pitch

    Remote multithreading stalls client ... and a pitch

    Hi,

    following the issue template: I expect the example below to work ^^ Problematic is the assignment process of incoming requests to local threads.

    To make it work rpyc obviously needs to spawn client side threads to "mirror" server side threads if necessary. What about

    • on request, threads enter a pool and send their request
      • requests contain two ids: (local thread id, remote thread id or None)
    • one thread in the pool is responsible for receiving
    • the receiving thread unpacks the request and directs it to the corresponding thread or a new thread
      • if the request is directed to itself, it will pass responsibility of receiving to another thread
    • the corresponding thread leaves the thread pool and processes the request
      • new threads may be reused

    This scheme has a positive side-effect: threads don't interfere with each other anymore (no race conditions, no stalling due to shared responsibilities). Downsides: more bytes/request and some use cases may create many idle threads (yet one thread sync request on many connections is fine). Async is covered by not entering the pool in the first place.

    Environment
    • rpyc 5.1.0
    • python 3.10
    • Arch Linux
    Minimal example

    Server:

    import rpyc
    import threading
    import time
    
    class Service(rpyc.Service):
        def exposed_function(self, event):
            threading.Thread(target=event.wait).start()
            time.sleep(1)
            threading.Thread(target=event.set).start()
    
    rpyc.ThreadedServer(Service(), port=18812).start()
    

    Client:

    import rpyc
    import threading
    
    connection = rpyc.connect("localhost", 18812, config=dict(allow_public_attrs=True))
    connection.root.function(threading.Event())
    
    Triage 
    opened by notEvil 23
  • Traceback with

    Traceback with "rpyc.classic.connect()"

    I am using python provided rpyc module for my remote connectivity purposes but my command "import rpyc; conn = rpyc.classic.connect()" yield the following traceback:

    [[email protected] masher]# python
    Python 2.7.5 (default, Aug  4 2017, 00:39:18)
    [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import rpyc
    >>> conn = rpyc.classic.connect("10.206.110.155")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/utils/classic.py", line 70, in connect
        return factory.connect(host, port, SlaveService, ipv6 = ipv6, keepalive = keepalive)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/utils/factory.py", line 93, in connect
        return connect_stream(s, service, config)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/utils/factory.py", line 54, in connect_stream
        return connect_channel(Channel(stream), service=service, config=config)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/utils/factory.py", line 43, in connect_channel
        return service._connect(channel, config)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/core/service.py", line 100, in _connect
        self.on_connect(conn)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/core/service.py", line 195, in on_connect
        self._install(conn, conn.root)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/core/service.py", line 199, in _install
        modules = ModuleNamespace(slave.getmodule)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/core/netref.py", line 157, in __getattribute__
        return syncreq(self, consts.HANDLE_GETATTR, name)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/core/netref.py", line 77, in syncreq
        return conn.sync_request(handler, proxy, *args)
      File "/usr/lib/python2.7/site-packages/rpyc-3.4.4-py2.7.egg/rpyc/core/protocol.py", line 516, in sync_request
        raise obj
    KeyError: '<rpyc.core.service.SlaveService object at 0x7fbfe0012090>'
    
    ========= Remote Traceback (1) =========
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/rpyc/core/protocol.py", line 347, in _dispatch_request
        res = self._HANDLERS[handler](self, *args)
      File "/usr/lib/python2.7/site-packages/rpyc/core/protocol.py", line 630, in _handle_getattr
        return self._access_attr(oid, name, (), "_rpyc_getattr", "allow_getattr", getattr)
      File "/usr/lib/python2.7/site-packages/rpyc/core/protocol.py", line 588, in _access_attr
        obj = self._local_objects[oid]
      File "/usr/lib/python2.7/site-packages/rpyc/lib/colls.py", line 86, in __getitem__
        return self._dict[key][0]
    KeyError: <rpyc.core.service.SlaveService object at 0x7fbfe0012090>
    
    >>>
    
    Environment
    • rpyc version = 3.4.4
    • python version = 2.7
    • operating system = CentOS-7.4 with kernel upgraded to 4.13.12

    Can anyone please help me rectify this issue?

    opened by naveenshankar61 23
  • recommend use of a decorator rather than expose_*

    recommend use of a decorator rather than expose_*

    rpyc: absolutely awesome.

    recommendation: decorator that exposes properties explicitly. this was a technique used in the pyjs jsonrpc service (which hilariously was 20 lines... gosh shock gasp 50 if you wanted remote error handling)

    opened by delandtj 19
  • Issues with multithreading and async, getting stuck

    Issues with multithreading and async, getting stuck

    I'm trying to use async with threads in order to not block my threaded server/service.

    There's an exposed function in the server that executes a closure in a thread, that function receives a callback function from the client, which is called when the closure returns.

    The client calls the function in several threads, each one with it's own callback function and Event object

    This works, but eventually gets stuck from some reason. it works on IronPython though without getting stuck.

    See this gist: https://gist.github.com/TI-AviBerko/fc6c09d187c764a1be3e19a5de7864e1

    opened by TI-AviBerko 13
  • AttributeError: characters_written

    AttributeError: characters_written

    I've got AttributeError when exceptions are raised:

    Client side:

    r=rpyc.connect("172.16.5.66", port=6666)
    P=r.root.Popen
    P("testtesttest")
    
    [skipped]
    /usr/lib/python3.3/site-packages/rpyc/core/stream.py in write(self, data)
        184         try:
        185             while data:
    --> 186                 count = self.sock.send(data[:self.MAX_IO_CHUNK])
        187                 data = data[count:]
        188         except socket.error:
    
    /usr/lib/python3.3/site-packages/rpyc/core/stream.py in __getattr__(self, name)
         66         if name.startswith("__"): # issue 71
         67             raise AttributeError("stream has been closed")
    ---> 68         raise EOFError("stream has been closed")
         69     def close(self):
         70         pass
    
    EOFError: stream has been closed
    

    Server side:

    Traceback (most recent call last):
      File "/usr/lib/python3.3/site-packages/rpyc/core/protocol.py", line 300, in _dispatch_request
        res = self._HANDLERS[handler](self, *args)
      File "/usr/lib/python3.3/site-packages/rpyc/core/protocol.py", line 532, in _handle_call
        return self._local_objects[oid](*args, **dict(kwargs))
      File "/home/sources/slave.py", line 36, in exposed_Popen
        p = MyPopen(cmd, **kwargs)
      File "/usr/lib/python3.3/subprocess.py", line 818, in __init__
        restore_signals, start_new_session)
      File "/usr/lib/python3.3/subprocess.py", line 1416, in _execute_child
        return self._local_objects[oid](*args, **dict(kwargs))
      File "/home/sources/slave.py", line 36, in exposed_Popen
        p = MyPopen(cmd, **kwargs)
      File "/usr/lib/python3.3/subprocess.py", line 818, in __init__
        restore_signals, start_new_session)
      File "/usr/lib/python3.3/subprocess.py", line 1416, in _execute_child
        raise child_exception_type(errno_num, err_msg)
    FileNotFoundError: [Errno 2] No such file or directory: 'testtesttest'
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "/usr/lib/python3.3/threading.py", line 639, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.3/threading.py", line 596, in run
        self._target(*self._args, **self._kwargs)
      File "/usr/lib/python3.3/site-packages/rpyc/utils/server.py", line 168, in _authenticate_and_serve_client
        self._serve_client(sock2, credentials)
      File "/usr/lib/python3.3/site-packages/rpyc/utils/server.py", line 194, in _serve_client
        conn.serve_all()
      File "/usr/lib/python3.3/site-packages/rpyc/core/protocol.py", line 395, in serve_all
        self.serve(0.1)
      File "/usr/lib/python3.3/site-packages/rpyc/core/protocol.py", line 386, in serve
        self._dispatch(data)
      File "/usr/lib/python3.3/site-packages/rpyc/core/protocol.py", line 354, in _dispatch
        self._dispatch_request(seq, args)
      File "/usr/lib/python3.3/site-packages/rpyc/core/protocol.py", line 311, in _dispatch_request
        self._send_exception(seq, t, v, tb)
      File "/usr/lib/python3.3/site-packages/rpyc/core/protocol.py", line 237, in _send_exception
        include_local_traceback = self._config["include_local_traceback"])
      File "/usr/lib/python3.3/site-packages/rpyc/core/vinegar.py", line 71, in dump
        attrval = getattr(val, name)
    AttributeError: characters_written
    

    Server-side script: https://github.com/kopchik/kvmtests/blob/master/slave.py

    opened by kopchik 13
  • __call__ always accessible, even when not exposed by _rpyc_getattr

    __call__ always accessible, even when not exposed by _rpyc_getattr

    RPYC has no mechanism to preclude access to the python __call__ magic method.

    Fetching a __call__ via __getattribute__ on a netref fetches the local netref __call__ function automatically: https://github.com/tomerfiliba/rpyc/blame/master/rpyc/core/netref.py#L148

    Then the __call__ method does a syncreq(_self, consts.HANDLE_CALL, args, kwargs) directly. https://github.com/tomerfiliba/rpyc/blame/master/rpyc/core/netref.py#L197

    This is sent on through the protocol, and there is no checking on either side if the remote __call__ attribute was exposed by _rpyc_getattr.

    Simple test case attached: rpyc__call__issue.zip

    Users of restricted() may get a nasty surprise. As for my own purposes, I am using rpyc and I have created a lot of tools and magic python decorators (which I may want to contribute back to rpyc) to describe security restrictions.

    I have RPYC configured so it doesn't allow ANYTHING to be accessed except via _rpyc_getattr. I was hoping that would include the __call__'s on class instances, classes themselves, and even function/method references. If there is no way enforce this, I will need to create my own patched fork of rpyc to meet my security model constraints.

    Since this 'hole" is longstanding, I think the best way to address it is with a new protocol_config flag that locks down all calling access (unless the __call__ attribute is exposed by _rpyc_getattr).

    I can create a patch myself, but want some feedback from maintainers before I go off and do that.

    opened by seveirein 12
  • Please create new version

    Please create new version

    Hi!

    since master is pretty stable (1 year without any commit), can we create new package on pypi? there are ~10 PR merged since last package creation

    Many thanks!

    opened by eplaut 12
  • Race condition in utils/factory:ssh_connect

    Race condition in utils/factory:ssh_connect

    I've found out that simultaneous rpyc.classic.ssh_connect calls from different instances could possibly lead to race condition in https://github.com/tomerfiliba/rpyc/blob/master/rpyc/utils/factory.py#L159 .

    Due to non-atomic operations of port discovery and port bind it could be possible to connect to another tunnel. ssh only warns you about port has been already used:

    bind: Address already in use
    channel_setup_fwd_listener: cannot listen to port: 2222
    

    Since ssh does not accept 0 as a bind port, I have no idea how to fix it easy.

    opened by yottatsa 12
  • OverflowError: Python int too large to convert to C long

    OverflowError: Python int too large to convert to C long

    reported by rudiger. here's a code snippet to reproduce the problem:

    
    >>> import rpyc
    >>> c=rpyc.classic.connect("pro114")
    
    >>> import platform
    >>> platform.architecture()   # client is 32 bit ubuntu 11.04, python 2.7
    ('32bit', 'ELF')
    >>> c.modules.platform.architecture()   # server is 64 bit generic linux, python 2.5
    ('64bit', 'ELF')
    
    >>> c.execute("""def f(lst):
    ...     for x in lst[1:]:
    ...             print x
    ... """)
    
    >>> l=[5,6,7,8,9]
    >>> c.namespace["f"]
    
    
    >>> c.namespace["f"](l)
    ======= Remote traceback =======
    Traceback (most recent call last):
      File "/usr/lib/python2.7/dist-packages/rpyc/core/protocol.py", line 227, in _dispatch_request
        res = self._HANDLERS[handler](self, *args)
      File "/usr/lib/python2.7/dist-packages/rpyc/core/protocol.py", line 445, in _handle_callattr
        return self._handle_getattr(oid, name)(*args, **dict(kwargs))
    OverflowError: Python int too large to convert to C long
    
    --------------------------------
    
    Traceback (most recent call last):
      File "rpyc/core/protocol.py", line 227, in _dispatch_request
      File "rpyc/core/protocol.py", line 433, in _handle_call
      File "", line 2, in f
      File "rpyc/core/netref.py", line 131, in method
      File "rpyc/core/netref.py", line 42, in syncreq
      File "rpyc/core/protocol.py", line 347, in sync_request
    OverflowError: Python int too large to convert to C long
    
    ======= Local exception ========
    Traceback (most recent call last):
      File "", line 1, in 
      File "/usr/lib/python2.7/dist-packages/rpyc/core/netref.py", line 125, in __call__
        return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
      File "/usr/lib/python2.7/dist-packages/rpyc/core/netref.py", line 42, in syncreq
        return conn().sync_request(handler, oid, *args)
      File "/usr/lib/python2.7/dist-packages/rpyc/core/protocol.py", line 347, in sync_request
        raise obj
    OverflowError: Python int too large to convert to C long
    

    very weird.

    Bug 
    opened by tomerfiliba 12
  • EOFerror stream has been closed

    EOFerror stream has been closed

    pretty new to python so sorry if this bug is dumb. I wrote a program that starts an arbitrary job from a set of python files from a directory - try_start_job is called with a job object, which has a first-class reference to the behavior to run. a new thread is spawned running an executor method, which calls the function to do the job along with some parameters from the job object. this works fine when the try_start_job method is called from within the same file it's defined, but when I try to execute the call to try_start_job over rpyc (trying to make a client-service-type-structure) I get the following error:

    Exception in thread ping:
    Traceback (most recent call last):
      File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
        self.run()
      File "/usr/lib/python3.5/threading.py", line 862, in run
        self._target(*self._args, **self._kwargs)
      File "/floodgate/libs/executor.py", line 18, in executor
        for _ in range(job.cycles):
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/netref.py", line 151, in __getattribute__
        return syncreq(self, consts.HANDLE_GETATTR, name)
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/netref.py", line 72, in syncreq
        return conn.sync_request(handler, oid, *args)
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/protocol.py", line 519, in sync_request
        self.sync_recv_and_dispatch(timeout, True)
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/protocol.py", line 414, in sync_recv_and_dispatch
        data = self._recv(timeout, wait_for_lock = False)
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/protocol.py", line 387, in _recv
        if self._channel.poll(timeout):
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/channel.py", line 43, in poll
        return self.stream.poll(timeout)
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/stream.py", line 40, in poll
        p.register(self.fileno(), "r")
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/stream.py", line 201, in fileno
        return self.sock.fileno()
      File "/usr/local/lib/python3.5/dist-packages/rpyc/core/stream.py", line 89, in fileno
        raise EOFError("stream has been closed")
    EOFError: stream has been closed
    

    (ping is the name of the thread - the test job just writes a line to stdout) I call the function from the 'client' like this:

    import rpyc
    conn = rpyc.connect("localhost", 18861, config={"allow_all_attrs": True})
    service = conn.root
    r = rpyc.async(service.try_start_job)
    res = r(testjob)
    print(res.value)
    

    what am I doing wrong?

    rpyc 3.4.4 python 3.5.2 Ubuntu 16.04.3 LTS

    opened by spacerainbow000 11
  • Fix race condition in AsyncResult.wait

    Fix race condition in AsyncResult.wait

    Fixes https://github.com/tomerfiliba-org/rpyc/issues/354 (and maybe more)

    • The receive lock is extended in both directions to cover AsyncResult.wait
    • data is split to allow lock release as soon as possible

    I can consistently reproduce issue 354 and provide an example if requested.

    Triage 
    opened by notEvil 10
  • How to execute exposed functions on my local machine?

    How to execute exposed functions on my local machine?

    I want to put the functions I wrote on the server for other machines or projects to call, but some functions must be executed locally. For example, I have a function that traverses local files or fetch localhost info. This function can only be executed locally; how can this function be placed on the server side? Or how can I achieve my needs with rpyc.

    Thanks a lot for your reply

    opened by ZKeeer 0
  • Strange interaction between RPyC and pyqtgraph

    Strange interaction between RPyC and pyqtgraph

    Using Python 3.11 and RPyC 5.3.0, I see a very puzzling interaction when talking to an RPyc server from a client that uses a pyqtgraph PlotWindow widget. (Pyqtgraph is a well-known framework for making plots in python programs that implement a Qt-based GUI).

    I am testing under Linux, with PyQt5 but the issue is also present when using PySide6.

    The server code is as plain and simple as can be:

    #! /usr/bin/env python3
    
    import logging
    import rpyc
    
    # Configure the logging infrastructure.
    logging.basicConfig(level=logging.DEBUG)
    
    class MyService(rpyc.Service):
        def __init__(self):
            super().__init__()
            self.exposed_value = 42
    
    server = rpyc.utils.server.ThreadedServer(MyService(), port=30000)
    server.start()
    

    The client side is somewhat more involved. I open a connection to the server and interact with it. Next, I run a simple GUI application that stops when the user closes the window. Finally, I close the connection:

    #! /usr/bin/env python3
    
    import sys
    import logging
    import time
    
    import rpyc
    
    from PyQt5.QtWidgets import QApplication, QWidget
    import pyqtgraph as pg
    
    # Configure the logging infrastructure.
    logging.basicConfig(level=logging.DEBUG)
    
    # Connect to the server's MyService instance.
    connection = rpyc.connect("localhost", 30000)
    
    # Obtain a value.
    # If we don't do this, the server-side issue disappears.
    
    value = connection.root.value
    print(value)
    
    # Make a trivial Qt application and run it.
    app = QApplication(sys.argv)
    main_window = pg.PlotWidget()  # If we use this, the server will emit an error upon closing the window.
    #main_window = QWidget()       # If we use this instead, the server will be happy.
    main_window.show()
    exitcode = app.exec()
    
    # As soon as we return from the application, the server-side generates a strange debug message.
    
    # Wait a bit (to establish when the server-side error is emitted), then close the connection.
    time.sleep(10)
    connection.close()
    

    The issue is that as soon as I close the GUI application, the server-side emits a debugging message that strongly suggests something has gone wrong. This is before the connection is closed:

    [email protected]:~/rpc_pyqtgraph_issue$ ./server.py 
    INFO:MY/30000:server started on [0.0.0.0]:30000
    INFO:MY/30000:accepted ('127.0.0.1', 43422) with fd 4
    INFO:MY/30000:welcome ('127.0.0.1', 43422)
    DEBUG:MY/30000:Exception caught
    Traceback (most recent call last):
      File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 356, in _dispatch_request
        res = self._HANDLERS[handler](self, *args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 853, in _handle_getattr
        return self._access_attr(obj, name, (), "_rpyc_getattr", "allow_getattr", getattr)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 780, in _access_attr
        name = self._check_attr(obj, name, param)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 770, in _check_attr
        raise AttributeError(f"cannot access {name!r}")
    AttributeError: cannot access '__class__'
    DEBUG:MY/30000:Exception caught
    Traceback (most recent call last):
      File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 356, in _dispatch_request
        res = self._HANDLERS[handler](self, *args)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 853, in _handle_getattr
        return self._access_attr(obj, name, (), "_rpyc_getattr", "allow_getattr", getattr)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 780, in _access_attr
        name = self._check_attr(obj, name, param)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/sidney/local_python/root/lib/python3.11/site-packages/rpyc/core/protocol.py", line 770, in _check_attr
        raise AttributeError(f"cannot access {name!r}")
    AttributeError: cannot access '__class__'
    INFO:MY/30000:goodbye ('127.0.0.1', 43422)
    ^C
    WARNING:MY/30000:keyboard interrupt!
    INFO:MY/30000:server has terminated
    INFO:MY/30000:listener closed
    

    There's a couple of ways I can make this debugging noise disappear.

    (1) If I don't interact from the client with the server other than opening/closing it, the issue disappears.

    (2) If I use a plain QWidget rather than a pyqtgraph PlotWidget, the issue disappears.

    My conclusion is that there is some kind of a funky interaction between pyqtgraph and rpyc; but I am not well-enough versed in both their codebases to investigate any further -- so I am hoping someone here can reproduce the issue, and see what's going on.

    opened by sidneycadot 2
  • Recommended way of doing service cluster?

    Recommended way of doing service cluster?

    e.g. I want to deploy a cluster of some service A. and each service has its availability or ready expectation time info available in a center service for query. clients query for the shortest time of available service A, and connect to that server for that service. Is there a recommended way. The more transparent the better for clients.

    opened by davidleon 0
  • Async Coroutine was never awaited.  (Missing __await__ method)

    Async Coroutine was never awaited. (Missing __await__ method)

    Describe the issue briefly here, including:

    Testing out RpyC for the groundwork of a project. I've setup a custom service that runs Discord.py on a seperate thread while RpyC runs on the main thread. I expected, from studying the documentation on RpyC, that the async_ wrapper is used for remote execution of asynchronous functions, and that executing a function wrapped in async_ would produce, in my case, a text message sent on on a text channel in a Discord Server.

    The actual result is an error that is thrown service side stating that the "coroutine 'Messable.send' was never awaited".

    Steps to reproduce: Copy the server code into a script and run, obviously you'll need to fetch a key from the Discord Developer Console. Run the client code in a script, you'll need to create a Discord guild/server you to add the bot to as well as supply the correct guild id and channel id. the result should produce the same error I have.

    /home/johndoe/.local/lib/python3.10/site-packages/rpyc/core/protocol.py:362: RuntimeWarning: coroutine 'Messageable.send' was never awaited
      self._dispatch_request(seq, args)
    
    Environment
    • rpyc version: 5.2.3
    • python version: 3.10
    • operating system: Kali Linux
    Minimal example

    Server:

    from threading import Thread
    import rpyc
    from rpyc.utils.server import ThreadedServer
    import discord
    from discord.ext.commands import Bot, when_mentioned_or
    import asyncio
    
    client = discord.Client(intents=discord.Intents.all())
    
    class DiscordService(rpyc.Service):
        def __init__(self):
            super().__init__()
            loop = asyncio.get_event_loop()
            loop.create_task(client.start('key'))
            Thread(target=loop.run_forever).start()
            print('Started Client.')
    
        exposed_client = client
    
        @client.event
        async def on_ready():
            await client.get_channel(836607364832296990).send('This is from the service')
    
    
    t = ThreadedServer(DiscordService(), port=18861, protocol_config={'allow_public_attrs': True})
    print('Starting service.')
    t
    

    Client:

    import rpyc
    from rpyc.utils.helpers import _Async
    import discord
    import asyncio
    
    conn = rpyc.connect('localhost', port=18861)
    d_client: discord.Client = conn.root.exposed_client
    channel: discord.TextChannel = d_client.get_guild(412361411159785493).get_channel(836607364832296990)
    channel_send = rpyc.async_(channel.send)
    c_result = channel_send('testing')
    c_result.wait()
    

    For benefit of ease, the function I am attempting to execute clientside (the send function): https://discordpy.readthedocs.io/en/stable/api.html?highlight=textchannel#discord.TextChannel.send

    Feature Help Wanted To Start 
    opened by TheMasteredPanda 2
  • rpyc_classic.py and rpyc_registry.py files no longer in Python Scripts DIR

    rpyc_classic.py and rpyc_registry.py files no longer in Python Scripts DIR

    Hello, sorry this might be a stupid question/bug report.

    In RPyC 5.2.3 I can see the following two executables in the Python Scripts DIR:

    image

    In all previous version of RPyC I have used (5.0, 5.1, 4.x) these have always been python (.py) files. Can I just confirm if this change is expected?

    Thanks!

    • rpyc version: 5.2.3
    • python version: 3.9.7
    • operating system: Windows 10
    Triage 
    opened by another-salad 5
Releases(5.3.0)
  • 5.3.0(Nov 26, 2022)

    5.3.0

    Date: 2022-11-25

    • #515 Support for Python 3.11 is available after teleportation bug fix
    • #507 Experimental support for threading is added (default is disabled for now)
    • #516 Resolved server-side exceptions due to the logic for checking if a name is in ModuleNamespace
    • #511 Improved documentation on the life-cycle of a netref/proxy-object

    Thread binding logic is still very experimental. Feel free to provide feedback, contributions, or alternative design options.

    Source code(tar.gz)
    Source code(zip)
  • 5.2.3(Aug 4, 2022)

    5.2.3

    Date: 2022-08-03

    • #503 rpyc_classic.py and rpyc_registry.py are tracked by pyproject.toml and should resolve now. Moreover, they can now be resolved without their file suffixes as well.
    Source code(tar.gz)
    Source code(zip)
  • 5.2.1(Jul 31, 2022)

    5.2.1

    Date: 2022-07-30

    • #494 Added support for using decorators to expose methods (see #292)
    • #499 Allow BgServingThread serve and sleep intervals to be customized
    • #498 Avoid redefining hasattr_static on every check_attr` call
    • #489 Updated SSL context usage to avoid deprecated aspects and changes
    • #485 Add a configurable timeout on the zero deploy close method
    • #484 Fixed --mode CLI argument for rpyc_registry
    • #479 Fixed propagation of AttributeErrors raised by exposed descriptors
    • #476 Allow filtering by host on list_services
    • #493 and #502 Improved documentation and fixed typos
    • #492 Some work around race conditions but proper fix is rather involved (see #491)

    5.2.0 was skipped due to PyPi not allowing file name reuse

    Source code(tar.gz)
    Source code(zip)
    rpyc-5.2.1-py3-none-any.whl(66.42 KB)
    rpyc-5.2.1.tar.gz(57.97 KB)
  • 5.1.0(Feb 26, 2022)

    • Added types.MappingProxyType to builtin_types #470
    • Updated documentation #469
    • Fixed spradic dealock issues from wait within AsyncResult #463 and #455
    • Fixed chained Classic RPyC connections #460
    • Added ability to list Registry services #452
    • Fixed bug that prevented RPyC from running on systems without SSL #451
    • Fixed unexpected behavior with respect to auto_register #445
    • Fixed propagation of chunk_size parameter for download_dir #433
    Source code(tar.gz)
    Source code(zip)
  • 5.0.1(Jan 11, 2021)

  • 5.0.0(Dec 26, 2020)

    Backwards Incompatible:

    • RPyC 5.0.0 cannot teleport functions to earlier versions
    • Deprecated Python 2 support to coincide with it's EOL

    Improvements:

    • Server hostname default supports IPv4 and IPv6 by using the wildcard address #425
    • Added docker/docker-compose.yml for Python 3.6, 3.7, 3.8, 3.9, and 3.10 containers to improve local workflow
    • Fixed pickle failure on windows for connect_multiprocess and connect_thread #412
    • Fixed teleport function behavior for keyword-only arguments with default #422
    • Improved documentation on custom exception handling
    • Fixed IPv6 support for server #407
    • Added a simple asynchrounous service example #400
    Source code(tar.gz)
    Source code(zip)
    rpyc-5.0.0-2-py3-none-any.whl(67.09 KB)
  • 4.1.5(Apr 25, 2020)

    • Fixed mutable object used as kwarg for Server ctor
    • Corrections to teleport example
    • Lowered GIL-lock acquires for <64kb within channel sends to address slowness
    Source code(tar.gz)
    Source code(zip)
  • 4.1.4(Jan 30, 2020)

    • Merged 3.7 and 3.8 teleportatio compat enhancement #371
    • Fixed connection hanging due to namepack cursor #369
    • Fixed test dependencies and is_py_* for 3.9
    Source code(tar.gz)
    Source code(zip)
  • 4.1.3(Jan 26, 2020)

    • Performance improvements: #366 and #351
    • Merged fix for propagate_KeyboardInterrupt_locally #364
    • Fixed handling of exceptions for request callbacks #365
    • Partially fixed return value for netref.__class__ #355
    Source code(tar.gz)
    Source code(zip)
  • 4.1.2(Oct 3, 2019)

  • 4.1.1(Jul 27, 2019)

    Release RPyC 4.1.1

    • Fixed netref.class_factory id_pack usage per #339 and added test cases
    • Name pack casted in _unbox to fix IronPython bug. Fixed #337
    • Increased chunk size to improve multi-client response time and throughput of large data #329
    • Added warning to _remote_tb when the major version of local and remote mismatch (#332)
    • OneShotServer termination was fixed by WilliamBruneau (#343)

    Note

    • Known issue with 3.8 for CodeType parameters (may drop Python2 support first)
    Source code(tar.gz)
    Source code(zip)
    rpyc-4.1.1.tar.gz(55.05 KB)
    rpyc-4.1.1.zip(71.44 KB)
  • 4.1.0(May 25, 2019)

    Release RPyC 4.1.0

    • Added connection back-off and attempts for congested workloads
    • Fixed minor resource leak for ForkingServer
    • Cross-connection instance check for cached netref classes

    Note

    • Requests/replies are not compatible between >=4.1.0 and earlier versions
    Source code(tar.gz)
    Source code(zip)
zerorpc for python

zerorpc Mailing list: [email protected] (https://groups.google.co

zerorpc 3k Dec 28, 2022
RPyC (Remote Python Call) - A transparent and symmetric RPC library for python

RPyC (pronounced like are-pie-see), or Remote Python Call, is a transparent library for symmetrical remote procedure calls, clustering, and distribute

1.3k Jan 05, 2023