Feapder的管道扩展

Related tags

Miscellaneousfeapder_pipelines
Overview

FEAPDER 管道扩展

简介

此模块为feapder的pipelines扩展，感谢广大开发者对feapder的贡献

随着feapder支持的pipelines越来越多，为减少feapder的体积，特将pipelines提出，使用者可按需安装

管道

PostgreSQL

贡献者：沈瑞祥

联系方式：[email protected]

安装

pip install feapder_pipelines[pgsql]

使用

在feapder项目的setting.py中使用如下配置

# PostgreSQL
PGSQL_IP = 
PGSQL_PORT = 
PGSQL_DB = 
PGSQL_USER_NAME = 
PGSQL_USER_PASS = 

ITEM_PIPELINES = [
    "feapder_pipelines.pipelines.pgsql_pipeline.PgsqlPipeline"
]
细节 注：入库时 ON CONFLICT(key) 默认为id或通过如下sql查出来的第一个值 
select column_names from(
        select
            t.relname as table_name,
            i.relname as index_name,
            array_to_string(array_agg(a.attname), ', ') as column_names
        from
            pg_class t,
            pg_class i,
            pg_index ix,
            pg_attribute a
        where
            t.oid = ix.indrelid
            and i.oid = ix.indexrelid
            and a.attrelid = t.oid
            and a.attnum = ANY(ix.indkey)
            and t.relkind = 'r'
            and t.relname like '%'
        group by
            t.relname,
            i.relname
        order by
            t.relname,
            i.relname) as res
    where table_name = 'table_name';
You might also like...
Releases(v1.0.3)
Owner
boris
爬虫工程师、数据工程师
boris
GitHub Repository
Safely pass trusted data to untrusted environments and back.

ItsDangerous ... so better sign this Various helpers to pass data to untrusted environments and to get it back safe and sound. Data is cryptographical

The Pallets Projects 2.6k Jan 01, 2023
freeCodeCamp Scientific Computing with Python Project for Certification.

Time_Calculator_freeCodeCamp freeCodeCamp Scientific Computing with Python Project for Certification. Write a function named add_time that takes in tw

Rajdeep Mondal 1 Dec 23, 2021
Dungeon Dice Rolls is an aplication that the user can roll dices (d4, d6, d8, d10, d12, d20 and d100) and store the results in one of the 6 arrays.

Dungeon Dice Rolls is an aplication that the user can roll dices (d4, d6, d8, d10, d12, d20 and d100) and store the results in one of the 6 arrays.

Bracero 1 Dec 31, 2021
Python program to start your zoom meetings

zoomstarter Python programm to start your zoom meetings More about Initially this was a bash script for starting zoom meetings, but as i started devel

Viktor Cvetanovic 2 Nov 24, 2021
Modern robots.txt Parser for Python

Robots Exclusion Protocol Parser for Python Robots.txt parsing in Python. Goals Fetching -- helper utilities for fetching and parsing robots.txts, inc

Moz 176 Dec 16, 2022
A tool to help plan vacations with friends and family

Vacationer In Development A tool to help plan vacations with friends and family Deployment Requirements: NPM Docker Docker-Compose Deployment Instruct

JK 2 Oct 05, 2021
Powerful Assistant

Delta-Assistant Hi I'm Phoenix This project is a smart assistant This is the 1.0 version of this project I am currently working on the third version o

1 Nov 17, 2021
python scripts - mostly automation scripts

python python scripts - mostly automation scripts You can set your environment in various ways bash #!/bin/bash python - locally on remote host #!/bi

Enyi 1 Jan 05, 2022
A python script based on OpenCV-Python, you can automatically hang up the Destiny 2 Throne to get the Dawning Essence.

A python script based on OpenCV-Python, you can automatically hang up the Destiny 2 Throne to get the Dawning Essence.

1 Dec 19, 2021
JD扫码获取Cookie 本地版

JD扫码获取Cookie 本地版 请无视手机上的提示升级京东版本的提示! 下载链接 https://github.com/Zy143L/jd_cookie/releases 使用Python实现 代码很烂 没有做任何异常捕捉 但是能用 请不要将获取到的Cookie发送给任何陌生人 如果打开闪退 请使

Zy143L 420 Dec 11, 2022
A small script I made that takes any standard Decklist of magic the gathering cards and pulls all card images from scryfall at once!

A small script I made that takes any standard Decklist of magic the gathering cards and pulls all card images from scryfall at once!

15 Aug 26, 2022
Flexible constructor to create dynamic list of heterogeneous properties for some kind of entity

Flexible constructor to create dynamic list of heterogeneous properties for some kind of entity. This set of helpers useful to create properties like contacts or attributes for describe car/computer/

Django Stars 24 Jul 21, 2022
An improved version of the common ˙pacman -S˙

BetterPacmanLook An improved version of the common pacman -S. Installation I know that this is probably one of the worst solutions and i will be worki

1 Nov 06, 2021
This is a Blender 2.9 script for importing mixamo Models to Godot-3

Mixamo-To-Godot This is a Blender 2.9 script for importing mixamo Models to Godot-3 The script does the following things Imports the mixamo models fro

8 Sep 02, 2022
A simple script that shows important photography times. written in python.

A simple script that shows important photography times. written in python.

John Evans 13 Oct 16, 2022
Banking management project using Tkinter GUI in python.

Bank-Management Banking management project using Tkinter GUI in python. Packages required Tkinter - Tkinter is the standard GUI library for Python. sq

Anjali Kumawat 7 Jul 03, 2022
WMIC Serial Checker For Python

WMIC Serial Checker Follow me here: Discord | Github FR: A but éducatif seulement. EN: For educational purposes only. ❓ Informations FR: WMIC Serial C

AkaTool's 0 Apr 25, 2022
JHBuild is a tool designed to ease building collections of source packages, called “modules”.

JHBuild README JHBuild is a tool designed to ease building collections of source packages, called “modules”. JHBuild was originally written for buildi

GNOME Github Mirror 46 Nov 22, 2022
一个IDA脚本，可以检测出哈希算法（无论是否魔改常数）并生成frida hook 代码。

findhash 在哈希算法上，比Findcrypt更好的检测工具，同时生成Frida hook代码。 使用方法 把findhash.xml和findhash.py扔到ida plugins目录下 ida -edit-plugin-findhash 试图解决的问题 哈希函数的初始化魔数被修改 想快速

266 Dec 29, 2022
Heisenbridge a bouncer-style Matrix IRC bridge

Heisenbridge brings IRC to Matrix by creating an environment where every user connects to each network individually like they would with a traditional IRC bouncer

Toni Spets 152 Dec 28, 2022