A lobby boy will create a VPS server when you need one, and destroy it after using it.

Overview

Lobbyboy

What is a lobby boy? A lobby boy is completely invisible, yet always in sight. A lobby boy remembers what people hate. A lobby boy anticipates the client's needs before the needs are needed. A lobby boy is, above all, discreet to a fault.

--The Grand Budapest Hotel

Test codecov PyPI version Python version

This project is still under testing, it worked but may have bugs.

What is lobbyboy?

Well, lobbyboy is a ssh server. Yes, like sshd. But instead of spawn a new shell on the server like sshd, when you ssh to lobbyboy, lobbyboy will create a new server(VPS) from available providers(meaning to say, DigitalOcean, AWS, GCP, Vultr, etc), then redirect you to the newly created servers. Of course, if lobbyboy finds any servers available already, he will just ask if you want to enter the existing server, or still want to create a new one.

Key Features

  • talks in SSH2 protocol, no need to install any software of configs for client-side, just ssh to lobbyboy!
  • extendable provider: just implement 3 methods, then lobbyboy can work with any provider!
  • destroy the server when you no longer needed.
  • manage ssh keys for you

Installation

Install libkrb5-dev first, this is a dependency for gssapi support.

apt install libkrb5-dev

Install via pip:

pip install lobbyboy

Run server

First, generate a config file:

lobbyboy-config-example > config.toml
# Edit your config before running!

Run the server with:

lobbyboy-server -c config.toml

You can ssh to Lobbyboy now, if you keep the default user Gustave in default config. You can ssh to Lobbyboy via:

ssh [email protected] -p 12200
# Enter the default password "Fiennes"(without quotes)
Welcome to Lobbyboy 0.2.2!
There are 1 available servers:
  0 - Create a new server...
  1 - Enter vagrant lobbyboy-41 127.0.0.1 (0 active sessions)
Please input your choice (number):

You may want to change the password in config.toml or use a public key for authentication. The latter is recommended in a production environment.

Generate a key pair for authentication

Generate a key pair:

ssh-keygen -f lobbyboy_key

Add the content of lobbyboy_key.pub to the end of authorized_keys under [user.Gustave] table. Now you can ssh to the lobbyboy server via:

ssh [email protected] -i lobbyboy_key

Deployment

Lobbyboy is supposed to be a server daemon, so you can manage it by systemd/supervisord or put it into a docker.

Systemd Example

[Unit]
Description=Lobbyboy Server

[Service]
User=me
Group=me
ExecStart=/path/to/lobbyboy-server -c /path/to/lobbyboy/config.toml
Restart=on-failure
WorkingDirectory=/path/to/lobbyboy/

[Install]
WantedBy=multi-user.target

Run in Docker

# Generate a config file
docker run --rm ghcr.io/lobbyboy-ssh/lobbyboy lobbyboy-config-example > lobbyboy_config.toml
# Run the docker container
docker run -v `pwd`/lobbyboy_config.toml:/app/config.toml -p "12200:12200" -d ghcr.io/lobbyboy-ssh/lobbyboy

The lobbyboy server should be active on 12200 port and you can connect to it with

ssh [email protected] -p 12200

The default password for user Gustave is Fiennes. Please change it when you deployed it into production, and consider use ssh key to auth instead of password.

Providers

// TBD

Builtin Providers

Lobbyboy current support multiple Providers:

  • Vagrant (Need vagrant and virtualbox to be installed)
  • Footlosse, in another words, containers (Need footloose and docker to be installed)
  • DigitalOcean
  • Linode

Different Providers support different configs, please see the example config for more detail.

Vagrant Provider

Vagrant Provider won't cost you any money, vagrant is a software runs on your computer along with virtual machine providers, vagrant can provision and control your VM.

This provider can help you to create a new Vagrant instance when you login to Lobbyboy, and destroy the server when you no longer use it.

Supported Features:

  • Create new Vagrant instances
  • You can configure your VM via vagrantfile config (see the config example).

Footloose Provider

footloose can make your docker containers(or Firecracker with ignite) act like virtual machine, so you can ssh to it.

Supported feature:

  • Configurable base image
  • Create a docker container and redirect you in

DigitalOcean Provider

This Provider will create Droplet from DigitalOcean.

Supported Features:

  • Create a new ssh key every time create a droplet.
  • Ask user to input region/droplet size/image when creating.
  • User can save favorite Droplet region/size/image in configs to quick create.
  • Destroy droplet when it is not in use.

Linode Provider

This Provider will create Node from Linode.

Supported Features:

  • Create a new ssh key every time create a droplet.
  • Ask user to input region/node type/image when creating.
  • User can save favorite node region/type/image in configs to quick create.
  • Destroy node when it is not in use.

Please see configs to check available options.

Ignite(Firecracker) Provider

Supported Features:

  • Create a new Firecracker virtual machine
  • Destroy node when it is not in use.

Write Your Own Providers

Providers are VPS vendors, by writing new providers, lobbyboy can work with any VPS vendors.

To make a new Provider work, you need to extend base class `lobbyboy.provider.BaseProvider``, implement 2 methods:

    def create_server(self, channel: Channel) -> LBServerMeta:
        """
        Args:
            channel: paramiko channel

        Returns:
            LBServerMeta: server meta info
        """
        ...


    def destroy_server(self, meta: LBServerMeta, channel: Channel = None) -> bool:
        """
        Args:
            meta: LBServerMeta, we use this to locate one server then destroy it.
            channel: Note that the channel can be None.
                     If called from server_killer, channel will be None.
                     if called when user logout from server, channel is active.

        Returns:
            bool: True if destroy successfully, False if not.
        """
        ...

Then add your Provider to your config file.

Those 3 configs are obligatory, as lobbyboy has to know when should he destroy your spare servers. You can add more configs, and read them from self.provider_config from code, just remember to add docs about it :)

[provider.<your provider name>]
load_module = "lobbyboy.contrib.provider.<your provider module name>::<Provider Class>"
min_life_to_live = "1h"
bill_time_unit = "1h"

Publish Your Own Providers

// TBD

FAQ

Q: Can I use lobbyboy as a proxy, like adding it to my ProxyCommand in ssh config?

A: No. Lobbyboy works like a reverse proxy, meaning to say, for ssh client, it just like a ssh server(sshd maybe), ssh client get a shell from lobbyboy, and doesn't know if it is local shell or it is a nested shell which runs another ssh. (but you know it, right? :D )

I Want to Know More!

Comments
  • LBConfigProvider is inflexible

    LBConfigProvider is inflexible

    My original idea for the provider is that:

    A provider can be defined in a single file, a provider's exceptions, implementations, and config definitions, all can be defined in a single file.

    So that it would be easy to maintain. And also, a provider can be implemented outside lobbyboy's codebase. (let's say, one can upload his provider to pypi, and other users can install them via pip install lobbyboy-aws-ec2-provider, then lobbyboy can load it.

    And one can add custom configs to his provider.

    The current problem is, we have to update the LBConfigProvider if one wants to add more fields to his provider config. It is not possible to do so without updating lobbyboy's source code.

    opened by laixintao 7
  • UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data

    CRI [20211208-07:42:34.995] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.                                                                                                                                                                                                                          Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 247, in run
        self.user_using(server, proxy_subprocess)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 188, in user_using
        send_to_channel(self.channel, os.read(master_fd, 10240).decode(), suffix="")
    UnicodeDecodeError: 'utf-8' codec can't decode bytes in position 4093-4094: unexpected end of data
    DEB [20211208-07:42:35.003] thr=140672300570368 paramiko.transport:1819: EOF in transport thread
    
    opened by messense 5
  • Initialization Problems.

    Initialization Problems.

    I am really interested in your idea of lobbyboy. But I know nothing about python.

    So here is my problems during initialization, can you help me to figure it out?

    1. I configure my own data_dir but service start with an error. obbyboy.utils: Error when reading available_servers.json, [Errno 2] No such file or directory: '~/Software/lobbyboy/data/available_servers.json' Even I create the available_servers.json file, it does not work. Until I write {} to the file. But I'm not sure if this is ok for the lobbyboy service
    2. When I ssh with ssh [email protected] -p 12200, the logs shows an error below:
    ERR [20211119-17:40:41.664] thr=123145413386240 paramiko.transport: Unknown exception: too many values to unpack (expected 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: Traceback (most recent call last):
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/transport.py", line 2109, in run
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     handler(self.auth_handler, m)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/paramiko/auth_handler.py", line 525, in _parse_userauth_request
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     result = self.transport.server_object.check_auth_publickey(
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:   File "/usr/local/lib/python3.9/site-packages/lobbyboy/server.py", line 103, in check_auth_publickey
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport:     _, _key_pub = line.split(" ", 2)
    ERR [20211119-17:40:41.667] thr=123145413386240 paramiko.transport: ValueError: too many values to unpack (expected 2)
    

    I did some research but it's really difficult for me to understand python.

    So if you can give me a hand, that's would be nice.

    Thank you.

    opened by kimichen13 4
  • Use pre-commit as the linter

    Use pre-commit as the linter

    Close #48

    Install the commit hooks in dev:

    poetry run pre-commit install
    

    Run in CI: I recommend using pre-commit ci, it can upgrade the hooks and commit linter changes automatically. It is also blazing fast, e2e run time is < 10s

    BTW, the code is changed by the linter so this is better to merge after #55, otherwise there will be lots of conflicts

    opened by frostming 3
  • Recommended use pre-commit to make code style consistency

    Recommended use pre-commit to make code style consistency

    @laixintao @messense @frostming

    Recommended use pre-commit to make code style consistency, especially in the case of multi-person cooperation, what do you think?

    opened by luxiaba 3
  • Bump paramiko from 2.8.1 to 2.10.1

    Bump paramiko from 2.8.1 to 2.10.1

    Bumps paramiko from 2.8.1 to 2.10.1.

    Commits
    • 286bd9f Cut 2.10.1
    • 4c491e2 Fix CVE re: PKey.write_private_key chmod race
    • aa3cc6f Cut 2.10.0
    • e50e19f Fix up changelog entry with real links
    • 02ad67e Helps to actually leverage your mocked system calls
    • 29d7bf4 Clearly our agent stuff is not fully tested yet...
    • 5fcb8da OpenSSH docs state %C should also work in IdentityFile and Match exec
    • 1bf3dce Changelog enhancement
    • f6342fc Prettify, add %C as acceptable controlpath token, mock gethostname
    • 3f3451f Add to changelog
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • OSError: [Errno 9] Bad file descriptor

    OSError: [Errno 9] Bad file descriptor

    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.utils:127: user choose 1 for option There are 1 available servers:
    INF [20211208-07:44:51.056] thr=140672309044992 lobbyboy.socket_handle:79: user choose server input=1.
    DEB [20211208-07:44:51.057] thr=140672309044992 lobbyboy.contrib.provider.ignite:86: get ssh to server command for ignite: ['cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739']
    INF [20211208-07:44:51.057] thr=140672309044992 lobbyboy.socket_handle:103: ssh to server 2021-12-08-0739 127.0.0.1: cd dev_datadir/ignite/2021-12-08-0739 && ignite ssh 2021-12-08-0739
    INF [20211208-07:44:51.062] thr=140672309044992 lobbyboy.socket_handle:176: proxy subprocess created, pid=71892
    CRI [20211208-07:44:51.987] thr=140672309044992 lobbyboy.socket_handle:254: *** Socket thread error.
    Traceback (most recent call last):
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 252, in run
        self.cleanup(t, meta=lb_server, check_destroy=True)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 194, in cleanup
        self.remove_server_session(t, meta.server_name)
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in remove_server_session
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/code/lobbyboy/lobbyboy/socket_handle.py", line 223, in <lambda>
        active_session[server_name] = list(filter(lambda x: x.getpeername() != peer_name, sessions))
      File "/root/.cache/pypoetry/virtualenvs/lobbyboy-P-fHd6h2-py3.8/lib/python3.8/site-packages/paramiko/transport.py", line 1787, in getpeername
        return gp()
    OSError: [Errno 9] Bad file descriptor
    
    opened by messense 1
  • tests for need_destroy

    tests for need_destroy

    somehow some of my instance didn't get destroyed, I suspect there are some bugs in https://github.com/lobbyboy-ssh/lobbyboy/blob/41bba07e856e6ce31f7164232723e7440a469f18/lobbyboy/server_killer.py#L42

    but I didn't find any, let me add test cases later.

    opened by laixintao 3
  • Security check when lobbyboy server starts.

    Security check when lobbyboy server starts.

    Need to check:

    • user started lobbyboy with default username/password?
    • with the ssh host key https://github.com/lobbyboy-ssh/lobbyboy/blob/main/dev_datadir/ssh_host_rsa_key in git repo (middle man attack)

    If yes, then need to print warning information to stderr.

    related discussion: https://github.com/lobbyboy-ssh/lobbyboy/pull/32

    opened by laixintao 0
  • vultr provider support

    vultr provider support

    i want to add vultr provider and i found python-vultr, but seems that it hasn't been updated for a long time, and there are fatal bugs that can't be used normally, i write a new one pyvultr, can we use this to support vultr provider?

    opened by luxiaba 2
Releases(v0.4.0)
  • v0.4.0(Dec 11, 2021)

    What's Changed

    • fix typo of exmaple config file name. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/25
    • Fix typos and add codespell check to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • Fix the path of example config.toml by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/28
    • Support docker container (based on footloose) by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/30
    • ci: fix unittest running branch master-> main by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/36
    • Add instructions to run in docker by @frostming in https://github.com/lobbyboy-ssh/lobbyboy/pull/32
    • Add ignite provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/37
    • Revamp provider config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/41
    • Fix send_to_channel by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/44
    • Add support for enable/disable provider in config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/45
    • Add a test case for load_config by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/49
    • Add a test case for load_providers by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/50
    • Add coverage report to CI by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/51
    • update readme: new architecture picture, update new providers. by @laixintao in https://github.com/lobbyboy-ssh/lobbyboy/pull/46
    • Define a default provider if there is only one provider by @messense in https://github.com/lobbyboy-ssh/lobbyboy/pull/47

    New Contributors

    • @messense made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/26
    • @frostming made their first contribution in https://github.com/lobbyboy-ssh/lobbyboy/pull/28

    Full Changelog: https://github.com/lobbyboy-ssh/lobbyboy/compare/v0.3.0...v0.4.0

    Source code(tar.gz)
    Source code(zip)
Micro Data Lake based on Docker Compose

Micro Data Lake based on Docker Compose This is the implementation of a Minimum Data Lake

Abel Coronado 15 Jan 07, 2023
🐳 RAUDI: Regularly and Automatically Updated Docker Images

🐳 RAUDI: Regularly and Automatically Updated Docker Images RAUDI (Regularly and Automatically Updated Docker Images) automatically generates and keep

SecSI 534 Dec 29, 2022
Ajenti Core and stock plugins

Ajenti is a Linux & BSD modular server admin panel. Ajenti 2 provides a new interface and a better architecture, developed with Python3 and AngularJS.

Ajenti Project 7k Jan 03, 2023
A charmed operator for running PGbouncer on kubernetes.

operator-template Description TODO: Describe your charm in a few paragraphs of Markdown Usage TODO: Provide high-level usage, such as required config

Canonical 1 Dec 01, 2022
Wiremind Kubernetes helper

Wiremind Kubernetes helper This Python library is a high-level set of Kubernetes Helpers allowing either to manage individual standard Kubernetes cont

Wiremind 3 Oct 09, 2021
This repository contains code examples and documentation for learning how applications can be developed with Kubernetes

BigBitBus KAT Components Click on the diagram to enlarge, or follow this link for detailed documentation Introduction Welcome to the BigBitBus Kuberne

51 Oct 16, 2022
Convenient tool to manage multiple VMs at once using libvirt

Convenient tool to manage multiple VMs at once using libvirt Installing To install the tool and its dependencies: pip install -e . Getting completion

Cedric Bosdonnat 13 Nov 11, 2022
Manage your SSH like a boss.

--- storm is a command line tool to manage your ssh connections. features adding, editing, deleting, listing, searching across your SSHConfig. command

Emre Yılmaz 3.9k Jan 03, 2023
gunicorn 'Green Unicorn' is a WSGI HTTP Server for UNIX, fast clients and sleepy applications.

Gunicorn Gunicorn 'Green Unicorn' is a Python WSGI HTTP Server for UNIX. It's a pre-fork worker model ported from Ruby's Unicorn project. The Gunicorn

Benoit Chesneau 8.7k Jan 08, 2023
Organizing ssh servers in one shell.

NeZha (哪吒) NeZha is a famous chinese deity who can have three heads and six arms if he wants. And my NeZha tool is hoping to bring developer such mult

Zilin Zhu 8 Dec 20, 2021
Pulumi - Developer-First Infrastructure as Code. Your Cloud, Your Language, Your Way 🚀

Pulumi's Infrastructure as Code SDK is the easiest way to create and deploy cloud software that use containers, serverless functions, hosted services,

Pulumi 14.7k Jan 08, 2023
Get Response Of Container Deployment Kube with python

get-response-of-container-deployment-kube 概要 get-response-of-container-deployment-kube は、例えばエッジコンピューティング環境のコンテナデプロイメントシステムにおいて、デプロイ元の端末がデプロイ先のコンテナデプロイ

Latona, Inc. 3 Nov 05, 2021
Utilitaire de contrôle de Kubernetes

Utilitaire de contrôle de Kubernetes ** What is this ??? ** Every time we use a word in English our manager tells us to use the French translation of

Théophane Vié 9 Dec 03, 2022
Python job scheduling for humans.

schedule Python job scheduling for humans. Run Python functions (or any other callable) periodically using a friendly syntax. A simple to use API for

Dan Bader 10.4k Jan 02, 2023
Asynchronous parallel SSH client library.

parallel-ssh Asynchronous parallel SSH client library. Run SSH commands over many - hundreds/hundreds of thousands - number of servers asynchronously

1.1k Dec 31, 2022
Official Python client library for kubernetes

Kubernetes Python Client Python client for the kubernetes API. Installation From source: git clone --recursive https://github.com/kubernetes-client/py

Kubernetes Clients 5.4k Jan 02, 2023
A repository containing a short tutorial for Docker (with Python).

Docker Tutorial for IFT 6758 Lab In this repository, we examine the advtanges of virtualization, what Docker is and how we can deploy simple programs

Arka Mukherjee 0 Dec 14, 2021
Deploy a simple Multi-Node Clickhouse Cluster with docker-compose in minutes.

Simple Multi Node Clickhouse Cluster I hate those single-node clickhouse clusters and manually installation, I mean, why should we: Running multiple c

Nova Kwok 11 Nov 18, 2022
A declarative Kubeflow Management Tool inspired by Terraform

🍭 KRSH is Alpha version, so many bugs can be reported. If you find a bug, please write an Issue and grow the project together! A declarative Kubeflow

Riiid! 128 Oct 18, 2022
Emissary - open source Kubernetes-native API gateway for microservices built on the Envoy Proxy

Emissary-ingress Emissary-Ingress is an open-source Kubernetes-native API Gateway + Layer 7 load balancer + Kubernetes Ingress built on Envoy Proxy. E

Emissary Ingress 4k Dec 31, 2022