pyreports is a python library that allows you to create complex report from various sources

Overview

pyreports

pyreports

Codacy Badge CircleCI

pyreports is a python library that allows you to create complex reports from various sources such as databases, text files, ldap, etc. and perform processing, filters, counters, etc. and then export or write them in various formats or in databases.

Test package

To test the package, follow these instructions:

$ git clone https://github.com/MatteoGuadrini/pyreports.git
$ cd pyreports
$ python -m unittest discover tests

Install package

To install package, follow these instructions:

$ pip install pyreports #from pypi

$ git clone https://github.com/MatteoGuadrini/pyreports.git #from official repo
$ cd pyreports
$ python setup.py install

Why choose this library?

pyreports wants to be a library that simplifies the collection of data from multiple sources such as databases, files and directory servers (through LDAP), the processing of them through built-in and customized functions, and the saving in various formats (or, by inserting the data in a database).

How does it work

pyreports uses the tablib library to organize the data into Dataset object.

Simple report

I take the data from a database table, filter the data I need and save it in a csv file

import pyreports

# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')

# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()

# Filter data
error_login = pyreports.Executor(site_login)
error_login.filter([400, 401, 403, 404, 500])

# Save report: this is a FileManager object
output = pyreports.manager('csv', '/home/report/error_login.csv')
output.write(error_login.get_data())

Combine source

I take the data from a database table, and a log file, and save the report in json format

import pyreports

# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')
# Select another source: this is a FileManager object
mylog = pyreports.manager('file', '/var/log/httpd/error.log')

# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()
error_log = mylog.read()

# Filter database
error_login = pyreports.Executor(site_login)
error_login.filter([400, 401, 403, 404, 500])
users_in_error = set(error_login.select_column('users'))

# Prepare log
myreport = dict()
log_user_error = pyreports.Executor(error_log)
log_user_error.filter(list(users_in_error))
for line in log_user_error:
    for user in users_in_error:
        myreport.setdefault(user, [])
        myreport[user].append(line)

# Save report: this is a FileManager object
output = pyreports.manager('json', '/home/report/error_login.json')
output.write(myreport)

Report object

import pyreports

# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')
output = pyreports.manager('xlsx', '/home/report/error_login.xlsx', mode='w')

# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()

# Create report data
report = pyreports.Report(site_login, title='Site login failed', filters=[400, 401, 403, 404, 500], output=output)
# Filter data
report.exec()
# Save data on file
report.export()

ReportBook collection object

import pyreports

# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')

# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()

# Create report data
report_failed = pyreports.Report(site_login, title='Site login failed', filters=[400, 401, 403, 404, 500])
report_success = pyreports.Report(site_login, title='Site login success', filters=[200, 201, 202, 'OK'])
# Filter data
report_failed.exec()
report_success.exec()
# Create my ReportBook object
my_report = pyreports.ReportBook([report_failed, report_success])
# Save data on Excel file, with two worksheet ('Site login failed' and 'Site login success')
my_report.export(output='/home/report/site_login.xlsx')

Tools for dataset

This library includes many tools for handling data received from databases and files. Here are some practical examples of data manipulation.

import pyreports

# Select source: this is a DatabaseManager object
mydb = pyreports.manager('mysql', host='mysql1.local', database='login_users', user='dba', password='dba0000')

# Get data
mydb.execute('SELECT * FROM site_login')
site_login = mydb.fetchall()

# Most common error
most_common_error_code = pyreports.most_common(site_login, 'code')  # args: Dataset, column name
print(most_common_error_code)   # 200

# Percentage of error 404
percentage_error_404 = pyreports.percentage(site_login, 404)    # args: Dataset, filter
print(percentage_error_404)   # 16.088264794 (percent)

# Count every error code
count_error_code = pyreports.counter(site_login, 'code')  # args: Dataset, column name
print(count_error_code)   # Counter({200: 4032, 201: 42, 202: 1, 400: 40, 401: 38, 403: 27, 404: 802, 500: 3})

Official docs

In the following links there is the official documentation, for the use and development of the library.

Open source

pyreports is an open source project. Any contribute, It's welcome.

A great thanks.

For donations, press this

For me

paypal

For Telethon

The Telethon Foundation is a non-profit organization recognized by the Ministry of University and Scientific and Technological Research. They were born in 1990 to respond to the appeal of patients suffering from rare diseases. Come today, we are organized to dare to listen to them and answers, every day of the year.

Telethon

Adopt the future

Acknowledgments

Thanks to Mark Lutz for writing the Learning Python and Programming Python books that make up my python foundation.

Thanks to Kenneth Reitz and Tanya Schlusser for writing the The Hitchhiker’s Guide to Python books.

Thanks to Dane Hillard for writing the Practices of the Python Pro books.

Special thanks go to my wife, who understood the hours of absence for this development. Thanks to my children, for the daily inspiration they give me and to make me realize, that life must be simple.

Thanks Python!

You might also like...
A complex language with high level programming and moderate syntax.

zsq a complex language with high level programming and moderate syntax.

A ULauncher/Albert extension that supports currency, units and date time conversion, as well as a calculator that supports complex numbers and functions.
A ULauncher/Albert extension that supports currency, units and date time conversion, as well as a calculator that supports complex numbers and functions.

Ulauncher/Albert Calculate Anything Ulauncher/Albert Calculate Anything is an extension for Ulauncher and Albert to calculate things like currency, ti

solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain
solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain

solsim is the Solana complex systems simulator. It simulates behavior of dynamical systems—DeFi protocols, DAO governance, cryptocurrencies, and more—built on the Solana blockchain

Tindicators is a Python library to calculate the values of various technical indicators

Tindicators is a Python library to calculate the values of various technical indicators

Minos-python - A framework which helps you create reactive microservices in Python

minos-python Summary [TODO] Packages minos-microservice-aggregate minos-microser

This is a vscode extension with a Virtual Assistant that you can play with when you are bored or you need help..

VS Code Virtual Assistant This is a vscode extension with a Virtual Assistant that you can play with when you are bored or you need help. Its currentl

You can easily send campaigns, e-marketing have actually account using cash will thank you for using our tools, and you can support our Vodafone Cash +201090788026

*** Welcome User Sorry I Mean Hello Brother ✓ Devolper and Design : Mokhtar Abdelkreem ========================================== You Can Follow Us O

A simple service that allows you to run commands on the server using text

Server Text A simple flask service that allows you to run commands on the server/computer over sms. Think of it as a shell where you run commands over

EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

EasyBuild is a software build and installation framework that allows you to manage (scientific) software on High Performance Computing (HPC) systems in an efficient way.

Releases(v1.5.1)
  • v1.5.1(Sep 27, 2022)

  • v1.5.0(Aug 4, 2022)

    Release notes

    • Added cli module
    • Added reports cli
    • Added _getitem_ method on Report class
    • Added _delitem_ method on Report class
    • Added _getitem_ method on ReportBook class
    • Added _delitem_ method on ReportBook class
    • Added _contains_ on Executor class
    • Fix NoSQLManager creation into manager function
    • Fix print_data on Report class
    Source code(tar.gz)
    Source code(zip)
  • v1.4.0(Jun 27, 2022)

    Release notes

    • Added _bool_ method on Report class
    • Added _iter_ method on Report class
    • Added _bool_ method on ReportBook class
    • Added _iter_ method on Connection and File classes
    • Added _iter_ method on FileManager class
    • Added _iter_ method on DatabaseManager class
    • Added _getitem_ on Executor class
    • Added _delitem_ on Executor class
    • Fix name of attachment on send method of Report class
    • Fix write method on LogFile class
    Source code(tar.gz)
    Source code(zip)
  • v1.3.1(Apr 29, 2022)

  • v1.3.0(Apr 15, 2022)

    Release notes

    • Added NoSQLManager class; this class extend Manager class on the nosqlapi package
    • Added LogFile class; this class load a log file and read method accept regular expression
    • Added _bool_ and _repr_ method on File and Connection abstract classes
    • Fix documentation API section
    • Fix tests package
    • Fix CircleCi docker image
    Source code(tar.gz)
    Source code(zip)
  • v1.2.0(Aug 5, 2021)

    Release notes

    1.2.0

    Aug 5, 2021

    • Added fill_value argument on aggregate function; this value also is callable without arguments
    • Added send method on Report class; with this method you send report via email
    • Added send method on ReportBook class; with this method you send report via email
    • Fix *str* method on Report class
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(Jun 5, 2021)

    Release notes

    • Created abstract File class
    • Created TextFile class
    • Added _str_ method for pretty representation of Executor class
    • Added _repr_ method for representation of DatabaseManager class
    • Added _repr_ method for representation of FileManager class
    • Added _repr_ method for representation of LdapManager class
    • Fix documentation for new abstract File class
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0(May 26, 2021)

    Release note

    The first pyreports release is here!

    With this first release it is possible to create input/output management objects such as Managers, process data thanks to Executors and do both, thanks to Report and ReportBook objects.

    Full documentation is at your disposal: https://pyreports.readthedocs.io/en/latest/

    Source code(tar.gz)
    Source code(zip)
Owner
Matteo Guadrini aka GU
Simple is better than complex. Complex is better than complicated.
Matteo Guadrini aka GU
Python-Roadmap - Дорожная карта по изучению Python

Python Roadmap Я решил сделать что-то вроде дорожной карты (Roadmap) для изучения языка Python. Возможно, если найдутся желающие дополнять ее, модифиц

Ruslan Prokhorov 1.2k Dec 28, 2022
🎴 LearnQuick is a flashcard application that you can study with decks and cards.

🎴 LearnQuick is a flashcard application that you can study with decks and cards. The main function of the application is to show the front sides of the created cards to the user and ask them to guess

Mehmet Güdük 7 Aug 21, 2022
1. 네이버 카페 댓글을 빨리 다는 기능

naver_autoprogram 기능 설명 네이버 카페 댓글을 빨리 다는 기능 네이버 카페 자동 출석 체크 기능 동작 방식 카페 댓글 기능 기본 동작은 주기적인 스케쥴 동작으로 해당 카페 ID 와 특정 API 주소로 대상이 새글을 작성했는지 체크. 해당 대상이 새글 등

1 Dec 22, 2021
Arabic to Roman Converter in Python

Arabic-to-Roman-Converter Made together with https://github.com/goltaraya . Arabic to Roman Converter in Python. -Instructions: 1 - Make sure you have

Pedro Lucas Tomazeti Fernandes 6 Oct 28, 2021
An easy way to access the Scratch API!

The majority of people are likely here because they want to easily access the Scratch API!

rgantzos 0 May 04, 2022
VHDL to Discrete Logic on PCB Flow

PCBFlow Highly experimental set of scripts to transform a digital circuit described in a hardware description language (VHDL or Verilog) into a discre

Tim 77 Nov 04, 2022
Projeto job insights - Projeto avaliativo da Trybe do Bloco 32: Introdução à Python

Termos e acordos Ao iniciar este projeto, você concorda com as diretrizes do Código de Ética e Conduta e do Manual da Pessoa Estudante da Trybe. Boas

Lucas Muffato 1 Dec 09, 2021
Python-geoarrow - Storing geometry data in Apache Arrow format

geoarrow Storing geometry data in Apache Arrow format Installation $ pip install

Joris Van den Bossche 11 Mar 03, 2022
An ongoing curated list of frameworks, libraries, learning tutorials, software and resources in Python Language.

Python Development Welcome to the world of Python. An ongoing curated list of frameworks, libraries, learning tutorials, software and resources in Pyt

Paul Veillard 2 Dec 24, 2021
In this project, we are going to display the battery notification and the time left for the battery to drain out using the battery capacity value.

In this project, we are going to display the battery notification and the time left for the battery to drain out using the battery capacity value.

Ritoban Biswas 1 Dec 20, 2021
A minimalist starknet amm adapted from StarkWare's amm.

viscus • A minimalist starknet amm adapted from StarkWare's amm. Directory Structure contracts

Alucard 4 Dec 27, 2021
Simple utlity for sniffing decrypted HTTP/HTTPS traffic on a jailbroken iOS device into an HAR format.

Description iOS devices contain a hidden feature for sniffing decrypted HTTP/HTTPS traffic from all processes using the CFNetwork framework into an HA

83 Dec 25, 2022
An event-based script that is designed to improve your aim

Aim-Trainer Info: This is an event-based script that is designed to improve a user's aim. It was built using Python Turtle and the Random library. Ins

Ethan Francolla 4 Feb 17, 2022
Transpiles some Python into human-readable Golang.

pytago Transpiles some Python into human-readable Golang. Try out the web demo Installation and usage There are two "officially" supported ways to use

Michael Phelps 318 Jan 03, 2023
A program that lets you use your tablet's tilting to emulate an actual joystick on a Linux computer.

Tablet Tilt Joystick A program that lets you use your tablet's tilting to emulate an actual joystick on a Linux computer. It's called tablet tilt joys

1 Feb 07, 2022
Python solution of advent-of-code 2021

Advent of code 2021 Python solutions of Advent of Code 2021 written by Eric Bouteillon Requirements The solutions were developed and tested using Pyth

Eric Bouteillon 3 Oct 25, 2022
Coded in Python 3 - I make for education, easily clone simple website.

Simple Website Cloner - Single Page Coded in Python 3 - I make for education, easily clone simple website. How to use ? Install Python 3 first. Instal

Phạm Đức Thanh 2 Jan 13, 2022
Never see escaped bytes in output.

Uniout It makes Python print the object representation in readable chars instead of the escaped string. Example from pprint import pprint lang

Mosky Liu 156 Oct 21, 2022
My HA controller for veg and flower rooms

HAGrowRoom My HA controller for veg and flower rooms I will do my best to keep this updated as I change, add and improve. System heavily uses custom t

4 May 25, 2022
An Advent calendar of small programming puzzles for a variety of skill sets and skill levels.

Advent of Code 2021 The Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be sol

Evan Cope 0 Feb 13, 2022