π
stonks-o-fetcher
A Simple modular tool to fetch and parse data related to the stock market.
Getting started
For the moment the only source is this repository, so to get the program you have to clone it locally.
Requirements
Python >3.6
The program is tested only on a linux environment (WSL 1 and debian) but should technically work on windows too I think.
Installation
After cloning and entering the root of the project
pip install .
If you are not on python 3
python3 -m pip install .
This will make the program available on your system with the command
stonks-cli
First steps
On first startup you'll have to setup your settings, especially the output path.
There is some validation for fields, so if something is missing you'll see it.
You can use the default ~ to point the path to your home folder, so you can set the path to something like ~/stonks/ or whatever you like.
If you define a filename in your path, meaning that it ends with either .csv or .txt that's the filename it will use to output the data, otherwise the filename will be generated automatically from the settings.
β οΈ File checkingThere is not check on existing files yet, and that's on purpose, so if you specify a custom file name it will be overwritten at every execution.
It is recommended to not specify a filename and let the program do its thing. It is also strongly suggested to actually change te path to something familiar.
Controls
The important things are esplained in the program itself, and are mostly out of my control due dependecies, but:
- Menu navigation:
arrow keysandvim bindings - Confirm a value:
Enter - Multiselect when available
Space- alsoenterwill add the currently highlighted entry - Exit from an input field with no defaults: type an
empty spacethenenter - With default values you can press
enterto confirm it.
β οΈ Saving your settingsExiting the application with
ESCwill NOT save your settings. you have to use the main menu option to do so.
CLI and automation
As of version 0.6.0 the only way to use this program is through the interactive cli menus, but i'm planning on adding the option to launch it with arguments to automate the execution of the process, specify all the required paramenters through arguments and handle different settings files to easily automate the execution through multiple settings.
Contributing
A proper documentation will come later, but here's the gist if you want to contribute on new features.
Components
The project is meant to be easily expandable and flexible. There are two main type of components to consider:
- Source components
- Fetchers
- Parsers
- Writer components
The name are pretty self explanatory I think.
The whole system is already setup to be almost completely automated Each source handler is included in its own module (folder). The module, through the __init__.py has to export some values:
Fetcher- your fetcher class, inheriting fromFetcherBaseParser- your parser class, inheriting fromParserBasesource-string.Unique value identifying the source handled, can be everyhingfriendly_name-string. The text that appears on the CLIdescription-stringa brief description of the source. appears in the cli.
Writers are similar, but instead of Fetcher and Parser and source they must have:
Writer- your writer class, inheriting fromWriterBaseoutput_type-stringunique identifier for the class.
The rest of the attributes remain the same.
βΉοΈ You can look at the existing modules insidestonks/componentsto better understand
There's a manager component that is already set up to import all the valid modules from the components/handlers and components/writers folders, so when your module is ready it should work. Loading is done in the cli module, so that the app is actually empty by itself.
For a module to be valid it has to have the required
classesand at least thesource/output_type
Custom formatting
If you take a look at the existing components description you'll notice some strange formatting.
The CLI has a custom formatter - because i like colored crayons - to ease highlighing important words. Instead of the standard string.format that replaces the values, here we wrap the words into {} to specify formatting.
# {word:color}
# {word:style}
# {word:color|style}
text = 'This {word:blue} is blue!'
# > this word is blue! - with `word` in blue.
Formatting is done through termcolor, so valid values are the ones in their documentation.
As before, check existing modules to better understand.
β οΈ String content For the moment there are a few issues with the default implementation ofstring.formatthat catches various character, specifically the.and:that is used as our delimiter, for now Inserting these character in a block to format will cause problems.As a rule of thumb, if you write your description and when testing the cli the page doesn't load, it means that there's probably something wrong with the text there.
Testing
Testing is done with pytest and coverage.
You can start a full run with
coverage run -m pytest -v && coverare report -m
Or use whatever integration you like - I'm using vscode and its integrations.
There is an utils file with a bunch of function and a decorator class, used mainly as container for the functions. Most of the tests require at least one decorator if they are not testing for failures.
Pull Requests
Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. I'm trying to follow git flow specs to some degree, so eventually the PR toward develop please.
17 Jan 13, 2022
1 Feb 04, 2022
13 Dec 01, 2022
2 Nov 21, 2021
41 Dec 02, 2022
27 Dec 14, 2022
1.7k Jan 07, 2023
571 Mar 15, 2022
36 Aug 22, 2022
29 Nov 30, 2022
222 Jan 04, 2023
6 Dec 03, 2022
29 Aug 18, 2022
3 Jan 06, 2022
17 Feb 02, 2022
14 Jul 26, 2022
9 Oct 17, 2022
95 Dec 25, 2022
1 Nov 06, 2021
5 Sep 21, 2022