Command line parser for common log format (Nginx default).

Related tags

Command-line Toolspythonnginxcommonlogging
Overview

Logparser

Command line parser for common log format (Nginx default).

Usage

It counts most important data: referrers, operating systems, browsers and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

Install and update PIP packages.

pip install -U -r requirements.txt

Speed

  • logparser 24,249/s
  • GoAccess 6,234/s

Outputs

  • HTML output is based on Jinja2 templates. It can be improved as you see fit.
  • Console output for Subreply for a fews days in Sep, 2021:
----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week
Owner
Lucian Marin
Python developer and all things design.
Lucian Marin
GitHub Repository
Python CLI for accessing CSCI320 PDM Database

p320_14 Python CLI for accessing CSCI320 PDM Database Authors: Aidan Mellin Dan Skigen Jacob Auger Kyle Baptiste Before running the application for th

Aidan Mellin 1 Nov 23, 2021
A terminal UI dashboard to monitor requests for code review across Github and Gitlab repositories.

A terminal UI dashboard to monitor requests for code review across Github and Gitlab repositories.

Kyle Harrison 150 Dec 14, 2022
Python codecs extension featuring CLI tools for encoding/decoding anything

CodExt Encode/decode anything. This library extends the native codecs library (namely for adding new custom encodings and character mappings) and prov

Alex 210 Dec 30, 2022
A terminal client for connecting to hack.chat servers

A terminal client for connecting to hack.chat servers.

V9 2 Sep 21, 2022
slipit is a command line utility for creating archives with path traversal elements.

slipit is a command line utility for creating archives with path traversal elements. It is basically a successor of the famous evilarc utility with an extended feature set and improved base functiona

usd AG 35 Dec 23, 2022
A very simple python script to encode and decode PowerShell one-liners.

PowerShell Encoder A very simple python script to encode and decode PowerShell one-liners. I used Raikia's PowerShell encoder ALOT, but one day it wen

John Tear 5 Jul 29, 2022
Centauro - a command line tool with some network management functionality

Centauro Ferramenta de rede O Centauro é uma ferramenta de linha de comando com

1 Jan 01, 2022
The Pythone Script will generate a (.)sh file with reverse shell codes then you can execute the script on the target

Pythone Script will generate a (.)sh file with reverse shell codes then you can execute the script on the targetPythone Script will generate a (.)sh file with reverse shell codes then you can execute

Boy From Future 15 Sep 16, 2022
Project scoped command execution to just do your work

Judoka is a command line utility that lets you define project scoped commands and call them through their alias. It lets you just do (= judo) your work.

Eelke van den Bos 2 Dec 17, 2021
A python package to display progress of loops to the user

ProgressBars A python package to display progress of loops to the user. Installation This package can be installed using pip. pip install progressbars

Matthias 3 Jan 16, 2022
Konsave lets use save your KDE Plasma customizatios and restore them very easily!

Konsave (Save Plasma Customization) A CLI program that will let you save and apply your KDE Plasma customizations with just one command! Als

439 Jan 02, 2023
A cd command that learns - easily navigate directories from the command line

NAME autojump - a faster way to navigate your filesystem DESCRIPTION autojump is a faster way to navigate your filesystem. It works by maintaining a d

William Ting 14.5k Jan 03, 2023
Convert shellcode into :sparkles: different :sparkles: formats!

Bluffy Convert shellcode into ✨ different ✨ formats! Bluffy is a utility which was used in experiments to bypass Anti-Virus products (statically) by f

AD995 305 Dec 17, 2022
Simple CLI prompt for easy I/O with OpenAI's API

openai-cli-prompt Simple CLI prompt for easy I/O with OpenAI's API Quickstart Create a .env file with: OPENAI_API_KEY=Your OpenAI API Key Configure

Erik Nomitch 1 Oct 12, 2021
A Command Line Error Parser Built using Python.

"Stalk Overflow with debuggy" Error Parser Everything is done in Python so it's extremely easy to install and use. Supports Python 3. Debuggy is used

Derhnyel 22 Nov 10, 2022
Play Wordle Bot - Wordle Bot written in python

Wordle Bot A Bot written in python with a CL Interface to guess adn solve Wordle

Prashant 1 Feb 25, 2022
Tidier - a simple command line tool that helps you make your files tidy up

Tidier - a simple command line tool that helps you make your files tidy up

AmirMohammad Hosseini Nasab 8 Aug 16, 2022
⌨ Toward a more useful keyboard

Toward a more useful keyboard Steve Losh's Modern Space Cadet is an inspiration. It opened my eyes to the fact that there's a more useful keyboard hid

Jason Rudolph 1.7k Jan 01, 2023
'rl_UK' is an open-source command-line tool in Python for calculating the shortest path between BUS stop sequences in the UK

'rl_UK' is an open-source command-line tool in Python for calculating the shortest path between BUS stop sequences in the UK. As input files, it uses an ATCO-CIF file and 'OS Open Roads' dataset from

Nesh P. 0 Feb 16, 2022
Amazon Scraper: A command-line tool for scraping Amazon product data

Amazon Product Scraper: 2021 Description A command-line tool for scraping Amazon product data to CSV or JSON format(s). Requirements Python 3 pip3 Ins

49 Nov 15, 2021