A pretty quick and simple interface to paramiko SFTP

Overview

sftpretty

A pretty quick and simple interface to paramiko SFTP. Provides multi-threaded routines with progress notifications for reliable, asynchronous transfers. This is a Python3 optimized fork of pysftp with additional features & improvements.

  • Built-in retry decorator
  • Hash function for integrity checking
  • Improved local & remote directory mapping
  • Improved logging mechanism
  • More tests
  • Multi-threaded directory transfers
  • Progress notifications
  • Support for digests & kex connection options
  • Support for ED25519 & ECDSA keys
  • Support for private key passwords
  • Thread-safe connection manager

Example

from sftpretty import Connection


# Basic

with Connection('hostname', username='me', password='secret') as sftp:
    # Temporarily chdir to public/.
    with sftp.cd('public'):
        # Upload file to public/ on remote.
        sftp.put('/my/local/filename')
        # Download a remote file from public/.
        sftp.get('remote_file')


with Connection('hostname', private_key='~/.ssh/id_ed25519',
                            private_key_pass='secret') as sftp:
    # Upload local directory to remote_directory.
    sftp.put_d('/my/local', '/remote_directory')

    # Recursively download a remote_directory and save it to /tmp locally.
    sftp.get_r('remote_directory', '/tmp')


# Advanced

with Connection('hostname', username='me', password='secret') as sftp:
    # Upload local directory to remote_directory. On occurance of any
    # exception or child of, passed in the tuple, retry the operation.
    # Between each attempt increment a pause equal to backoff * delay.
    # Run a total of tries (six) times including the first attempt.
    sftp.put_d('/my/local', '/remote_directory',
               exceptions=(NoValidConnectionsError,
                           socket.timeout,
                           SSHException),
               tries=6, backoff=2, delay=1)


with Connection('hostname', private_key='~/.ssh/id_ed25519',
                            private_key_pass='secret') as sftp:
    # Recursively download a remote_directory and save it to /tmp locally.
    # Don't confirm files, useful in a scenario where the server removes
    # the remote file immediately after download. Preserve remote mtime on
    # local copy
    sftp.get_r('remote_directory', '/tmp', confirm=False,
               preserve_mtime=True)

Additional Information

Requirements

paramiko >= 1.17.0

Supports

Tested on Python 3.6, 3.7, 3.8, 3.9

https://travis-ci.org/byteskeptical/sftpretty.svg?branch=master
Comments
  • put_d() unexpected remote path handling and issue with absolute Windows paths

    put_d() unexpected remote path handling and issue with absolute Windows paths

    I found two issues with the put_d() function.

    1. It looks like it doesn't accept absolute Windows paths. I get the error:

    ValueError: 'C:\\Users\\johnm\\PycharmProjects\\project\\.other\\folder\\file' does not start with '\\'

    1. When I call put_d("local_folder", "folder"), I get the following error:

    File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\sftpretty\__init__.py", line 563, in put confirm=confirm, preserve_mtime=preserve_mtime) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\sftpretty\__init__.py", line 554, in _put confirm=confirm) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 759, in put return self.putfo(fl, remotepath, file_size, callback, confirm) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 714, in putfo with self.file(remotepath, "wb") as fr: File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 372, in open t, msg = self._request(CMD_OPEN, filename, imode, attrblock) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 822, in _request return self._read_response(num) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 874, in _read_response self._convert_status(msg) File "C:\Users\johnm\PycharmProjects\project\venv\lib\site-packages\paramiko\sftp_client.py", line 903, in _convert_status raise IOError(errno.ENOENT, text) FileNotFoundError: [Errno 2] Path not found.

    As I understand, the path it refers to is the remote path. When I debug it, in put_d(), the paths it creates are the following: image As you can see, the remotedir "folder/local_folder/file" is probably the issue. In my remote location, it creates the folder "folder". This of course makes sense as it successfully calls self.mkdir_p(remotedir) in line 599. Do you think this has to do with the location of the self.mkdir_p(remotedir) call? Should there be code that also creates the subfolders of the "local_folder" in the remote location(here the "folder/local_folder" subfolder)?

    Thank you for your time.

    bug 
    opened by Johnmaras 9
  • Add support to use a single channel for all commands. Some servers do…

    Add support to use a single channel for all commands. Some servers do…

    Certain SFTP server do not allow to open a channel for every command. The original pysftp used a single channel for all actions. I've added a flag to allow us to work in a similar way, while by default keeping the new behavior.

    Those servers will simply close the connection on the second command issued and it makes sftpretty not very useful with these old(er) SFTP servers.

    opened by erans 5
  • Typo in argument name - tires/tries

    Typo in argument name - tires/tries

    Hi,

    In sftpretty/init.py:716 there is a typo in the tries argument which causes the call to putfo() to raise an exception

    retry() got an unexpected keyword argument 'tires'

    bug 
    opened by Johnmaras 2
  • Fix potential `_channel` `UnboundLocalError`

    Fix potential `_channel` `UnboundLocalError`

    Sometimes when assigning to _channel, an exception can occur in SFTPClient.from_transport(). Because _channel doesn't exist if that happens, it causes a second exception when trying to call close() in the finally block. Assign to the variable outside the try block initially so that this exception can't happen in the future. Also make sure that _channel is not None before we try to close it to account for it being sometimes None.

    bug 
    opened by RobertCochran 1
  • calling basicConfig at this level for a library that should be embedded changes all dowstream logs

    calling basicConfig at this level for a library that should be embedded changes all dowstream logs

    Calling basicConfig on the top of the file for a library that is embedded into other projects can cause issue. I suggest you drop it completely specifically because there are no change to it other than setting the default level to INFO.

    https://github.com/byteskeptical/sftpretty/blob/ca8e2414ac156276059fcd091cc91e489d856e7d/sftpretty/init.py#L22

    In our case it added ":" to the start of every log line.

    enhancement 
    opened by erans 1
Releases(1.0.5)
Mini SCADA. Poll modbus devices by TCP/IP network.

Plans Add saving and loading devices and channels with files or db or someone else. Multitasking system for poll all devices Automatic optimization po

Efi_fi 1 Oct 25, 2021
Openconnect VPN RPi Gateway

Openconnect-VPN-RPi-Gateway See the blog (Chinese) for how to build an Openconne

Zhongze Tang 2 Jan 30, 2022
A simple tool to get information about IP

IP Info Tool Just a simple tool to get IP's information, it uses requests module to gather information about IP, if you dont have much knowledge about

0 Dec 01, 2021
Monitoring plugin to check network interfaces with Icinga, Nagios and other compatible monitoring solutions

check_network_interface - Monitor network interfaces This is a monitoring plugin for Icinga, Nagios and other compatible monitoring solutions to check

DinoTools 3 Nov 15, 2022
MoreIP 一款基于Python的面向 MacOS/Linux 用户用于查询IP/域名信息的日常渗透小工具

MoreIP 一款基于Python的面向 MacOS/Linux 用户用于查询IP/域名信息的日常渗透小工具

xq17 9 Sep 21, 2022
An automatic reaction network generator for reactive molecular dynamics simulation.

ReacNetGenerator An automatic reaction network generator for reactive molecular dynamics simulation. ReacNetGenerator: an automatic reaction network g

Tong Zhu Research Group 35 Dec 14, 2022
A TrueCharts automatic and bulk update utility

trueupdate A TrueCharts automatic and bulk update utility How to install run pip install trueupdate Please be aware you will need to reinstall after e

TrueCharts 125 Jan 04, 2023
A non-custodial oracle and escrow system for the lightning network. Make LN contracts more expressive.

Hodl contracts A non-custodial oracle and escrow system for the lightning network. Make LN contracts more expressive. If you fire it up, be aware: (1)

31 Nov 30, 2022
This is an open project to maintain a list of domain names that serve YouTube ads

The YouTube ads blocklist project This is an open project to maintain a list of domain names that serve YouTube ads. The original project only produce

Evan Pratten 574 Dec 30, 2022
Dokumentasi belajar Network automation

Repositori belajar network automation dengan Docker, Python & GNS3 Using Frameworks and integrate with: Paramiko Netmiko Telnetlib CSV SFTP Netmiko, S

Daniel.Pepuho 3 Mar 15, 2022
ARTEMIS: Real-Time Detection and Automatic Mitigation for BGP Prefix Hijacking.

ARTEMIS: Real-Time Detection and Automatic Mitigation for BGP Prefix Hijacking. This is the main ARTEMIS repository that composes artemis-frontend, artemis-backend, artemis-monitor and other needed c

INSPIRE Group @FORTH-ICS 273 Jan 01, 2023
WebRTC and ORTC implementation for Python using asyncio

aiortc What is aiortc? aiortc is a library for Web Real-Time Communication (WebRTC) and Object Real-Time Communication (ORTC) in Python. It is built o

3.2k Jan 07, 2023
This script helps us to add IP, host name entry in hosts file and create directory run nmap scan and directory scan with your favourite tools

A python script to automate your set-up for Hack The Box, It sets up Workspace, Opens TMUX session, connects to OpenVPN, Runs Nmap and many more...

Cognizance 7 Mar 25, 2022
wg-exporter is a simple yet effective Prometheus exporter for Wireguard

wg-exporter wg-exporter is a simple yet effective Prometheus exporter for Wireguard. What are the collected metrics ? General: wg_connected_peers: num

men1n2 3 May 20, 2022
Mass Reverse IP Dibuat Dengan Python 3 Dan Ada Fitur Filter.

Reverse IP Tools Description. Reverse IP is a method to map an IP address to a sub domain. This tool is made in the python 3 programming language. Fea

Wan Naz ID 6 Oct 24, 2022
TicTacToe using Socket Server

TicTacToe using Socket Server This is a project for the class : 18CSC302J - Computer Networks by Dr. S.Babu Contributors Suvodeep Sinha RA191100301010

Suvodeep Sinha 12 Nov 30, 2022
A collection of domains, wildcards and substrings designed for dnscrypt-proxy filter method.

A collection of domains, wildcards and substrings designed for dnscrypt-proxy filter method.

3 Oct 25, 2022
DEMO SOCKET AF INET SSL PYTHON

DEMO_SOCKET_AF_INET_SSL_PYTHON Python demo of socket family as AF_INET using TCP with SSL. Compatibility : macOS & GNU/Linux Network Topology style :

Enola 1 Jan 24, 2022
IPV4 network calculation project in Python

Curso de Python 3 do Básico ao Avançado Desafio: Calculando redes IPV4 Criar um programa que obtem um numero de IP com o prefixo da mascara de rede. O

Diego Guedes 3 Jan 21, 2022
An API for controlling Wi-Fi connections on Balena devices.

Description An API for controlling Wi-Fi connections on Balena devices. It does not contain an interface, instead it provides API endpoints to send re

8 Dec 25, 2022