Yaml - Loggers are like print() statements

Overview

Upgrade your print statements

print statement meme

Loggers are like print() statements

except they also include loads of other metadata:

  • timestamp
  • msg (same as print!)
  • args (values or variables put into functions)
  • function name
  • level (e.g. DEBUG)
  • line number (e.g line 42)
  • module (which python script it came from)
  • processes (getting fancy)
  • threads (getting very fancy)

I want it all WITHOUT CONFIGURATION!

loguru logo

No problems. Check out the Loguru repo!
You can pretty much run with minimal config and loads of features. Loguru has all the features of this repo and more, along with a badass logo.

Where do I start?

This repo comes bundled with tidy logging configuration files saved in src/logconfig/.
There is also a timing decorator in src/utils/ so you can optimize your code by simply decorating your function with @timing above it. There are many online articles but very few clearly explain how to configure complex loggers with filters and yaml files.
To see loggers in action, setup a virtual environment and then run src/mainmodule.py

Requirements

  • Python 3.6+

(Feel free to submit earlier versions that work)

Quickstart - Git clone and virtual env setup

Quickstart Instructions

Windows using powershell or CMD

cd to clone directory. Create virtual env with pip + venv:

git clone https://github.com/izzley/loggerexamples
cd loggerexamples\
py -0p # Optional: check your version and python path
py -m venv .venv
.venv\Scripts\activate
pip install --upgrade pip
pip install -r requirements.txt

# run main script
.\src\main.py

Linux/Mac

cd /to/clone/location
git clone https://github.com/izzley/loggerexamples
cd loggerexamples/
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

If your relative imports aren't working, create `.pth' and add the parent folder/s to the file:

$ echo $(pwd) >> .venv/lib/python3.8/site-packages/my_p_ext.pth

Play with the logger

Walkthroughs

@TODO add list of suggestions to inspect

Try changing the main logger level from main_module.py

The main logger is first setup from main_module.py. Notice how the level is set to DEBUG?

logger level change


  • Run main_module.py with this DEBUG setting and take a look at src/logconfig/root.log.
  • Now change the level to WARNING and spot the difference in src/logconfig/root.log. Remember this is the __main__ logger. Notice any changes?

logger level change


  • Notice everything from __main__ is greater or equal to WARNING? The logs running from other_module.py are still showing because we haven't changed its log level. Go ahead and change the loglevel in other_module.py to CRITICAL and spot the difference.

Whats in the YAML file??

YAML parts

In short, the conf.YAML file contains all of the instructions for how the logger should behave. Below breaks down the conf yaml file into its parts:

formatters

formatters:
    standard:
        format: "%(asctime)s %(levelname)s - [%(filename)s: line %(lineno)s] - %(funcName)s - %(message)s"

Take this logger for example:

def funccalc(n):
    logger.debug("something executed")
    for _ in range(n):
        i = 0
    return

The output reflects the yaml file format settings:

2021-11-21 15:43:47,689 DEBUG - [module01.py: line 17] - funccalc - something executed

loggers

@TODO describe root loggers and their inheritance

root:
  level: DEBUG
  handlers: [console, debug_file_handler, info_file_handler, warn_file_handler, error_file_handler, critical_file_handler, root_file_handler]

root logger yaml

handlers

@TODO describe handlers and how they redirect bytes

filters

@TODO describe how filters only allow bytes to handlers if a condition is true. reference filter classes in logconfig.py

References

Owner
isaac peterson
Open source || Awesome sauce
isaac peterson
Splunk Add-On to collect audit log events from Github Enterprise Cloud

GitHub Enterprise Audit Log Monitoring Splunk modular input plugin to fetch the enterprise audit log from GitHub Enterprise Support for modular inputs

Splunk GitHub 12 Aug 18, 2022
Multi-processing capable print-like logger for Python

MPLogger Multi-processing capable print-like logger for Python Requirements and Installation Python 3.8+ is required Pip pip install mplogger Manual P

Eötvös Loránd University Department of Digital Humanities 1 Jan 28, 2022
Stand-alone parser for User Access Logging from Server 2012 and newer systems

KStrike Stand-alone parser for User Access Logging from Server 2012 and newer systems BriMor Labs KStrike This script will parse data from the User Ac

BriMor Labs 69 Nov 01, 2022
HTTP(s) "monitoring" webpage via FastAPI+Jinja2. Inspired by https://github.com/RaymiiOrg/bash-http-monitoring

python-http-monitoring HTTP(s) "monitoring" powered by FastAPI+Jinja2+aiohttp. Inspired by bash-http-monitoring. Installation can be done with pipenv

itzk 39 Aug 26, 2022
Display tabular data in a visually appealing ASCII table format

PrettyTable Installation Install via pip: python -m pip install -U prettytable Install latest development version: python -m pip install -U git+https

Jazzband 924 Jan 05, 2023
Colored terminal output for Python's logging module

coloredlogs: Colored terminal output for Python's logging module The coloredlogs package enables colored terminal output for Python's logging module.

Peter Odding 496 Dec 30, 2022
GTK and Python based, system performance and usage monitoring tool

System Monitoring Center GTK3 and Python 3 based, system performance and usage monitoring tool. Features: Detailed system performance and usage usage

Hakan Dündar 649 Jan 03, 2023
A watchdog and logger to Discord for hosting ScPrime servers.

ScpDog A watchdog and logger to Discord for hosting ScPrime servers. Designed to work on Linux servers. This is only capable of sending the logs from

Keagan Landfried 3 Jan 10, 2022
Token Logger with python

Oxy Token Stealer Features Grabs discord tokens Grabs chrome passwords Grabs edge passwords Nothing else, I don't feel like releasing full on malware

oxy 1 Feb 12, 2022
Outlog it's a library to make logging a simple task

outlog Outlog it's a library to make logging a simple task!. I'm a lazy python user, the times that i do logging on my apps it's hard to do, a lot of

ZSendokame 2 Mar 05, 2022
Discord-Image-Logger - Discord Image Logger With Python

Discord-Image-Logger A exploit I found in discord. Working as of now. Explanatio

111 Dec 31, 2022
A Python package which supports global logfmt formatted logging.

Python Logfmter A Python package which supports global logfmt formatted logging. Install $ pip install logfmter Usage Before integrating this library,

Joshua Taylor Eppinette 15 Dec 29, 2022
Robust and effective logging for Python 2 and 3.

Robust and effective logging for Python 2 and 3.

Chris Hager 1k Jan 04, 2023
This is a key logger based in python which when executed records all the keystrokes of the system it has been executed on .

This is a key logger based in python which when executed records all the keystrokes of the system it has been executed on

Purbayan Majumder 0 Mar 28, 2022
Translating symbolicated Apple JSON format crash log into our old friends :)

CrashTranslation Translating symbolicated Apple JSON format crash log into our old friends :) Usage python3 translation.py -i {input_sybolicated_json_

Kam-To 11 May 16, 2022
APT-Hunter is Threat Hunting tool for windows event logs

APT-Hunter is Threat Hunting tool for windows event logs which made by purple team mindset to provide detect APT movements hidden in the sea of windows event logs to decrease the time to uncover susp

824 Jan 08, 2023
Command-line tool that instantly fetches Stack Overflow results when an exception is thrown

rebound Rebound is a command-line tool that instantly fetches Stack Overflow results when an exception is thrown. Just use the rebound command to exec

Jonathan Shobrook 3.9k Jan 03, 2023
Fuzzy-logger - Fuzzy project is here Log all your pc's actions Simple and free to use Security of datas !

Fuzzy-logger - ➡️⭐ Fuzzy ⭐ project is here ! ➡️ Log all your pc's actions ! ➡️ Simple and free to use ➡️ Security of datas !

natrix_dev 2 Oct 02, 2022
A simple package that allows you to save inputs & outputs as .log files

wolf_dot_log A simple package that allows you to save inputs & outputs as .log files pip install wolf_dot_log pip3 install wolf_dot_log |Instructions|

Alpwuf 1 Nov 16, 2021
Python script to scan log files/system for unauthorized access around system

checkLogs Python script to scan log files/system for unauthorized access around Linux systems Table of contents General info Getting started Usage Gen

James Kelly 1 Feb 25, 2022