Avoid PyCharm Overloading ========================= This speeds up PyCharm's package index processes and avoids CPU & memory overloading. On my machine (4c/32GB) with 5 conda environments and 10 projects, this reduces PyCharm's package scanning from 15 minutes to 45 seconds. Why? ---- When PyCharm has multiple conda environments, it will sometimes launch concurrent package scanning across those environments. To do so, it uses the `conda` cli, and a tool called conda_packaging_tool.py, available from the JetBrains intellij-community github repository (see below). Unfortunately, these tools do not behave well when launched in parallel, as they tend to overload CPU & memory in this case. How? ---- To avoid stalling these processes, we modify their sources as follows: 1. modify the conda cli $ conda activate base $ pip install runfast $ nano `which conda` Modify the main section: if __name__ == '__main__': from conda.cli import main from runfast import cached cached(main) 2. modify the conda_packaging_tool.py if __name__ == '__main__': from runfast import cached cached(main) How does this work? ------------------- Two simple steps: 1. When called in parallel, only one process is allowed to proceed. 2. `runfast.cached` caches the output of these tools (stdout, stderr) for 1 minute, given the same command line parameters That is, when PyCharm launches 5 package scanning (`conda list`) commands, only one of them will run immediately. If some of the scans are for the same environment, only one of them will actually run, while the others simply return the same output. How to clear the cache or avoid caching at all? $ export RUNFAST_NOCACHE=1 Testing ------- a) conda packaging tool # first time $ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py real 0m13.271s user 0m9.631s sys 0m1.936s # second time $ time python /opt/pycharm-community-2021.2.1/plugins/python-ce/helpers/conda_packaging_tool.py real 0m3.159s user 0m0.139s sys 0m1.298s b) conda cli # first time $ time conda list -p /path/to/env real 0m7.985s user 0m7.883s sys 0m0.087s # second time real 0m0.152s user 0m0.116s sys 0m0.032s References ---------- * https://github.com/JetBrains/intellij-community/blob/master/python/helpers/conda_packaging_tool.py * https://github.com/conda/conda/blob/33a142c16530fcdada6c377486f1c1a385738a96/conda/core/index.py#L53
This speeds up PyCharm's package index processes and avoids CPU & memory overloading
Overview
Convex Optimisation MVA course - Assignment
Convex Optimisation MVA course - Assignment This repository contains the coding files of the third assignment in the MVA Convex Optimisation course. U
An example using debezium and mysql with docker-compose
debezium-mysql An example using debezium and mysql with docker-compose The docker compose starts the Zookeeper, Kafka, Mysql and Debezium Connect. Aft
A curses based mpd client with basic functionality and album art.
Miniplayer A curses based mpd client with basic functionality and album art. After installation, the player can be opened from the terminal with minip
CPLib is the abbreviation of Competitive Programming Library.
CPLib CPLib is the abbreviation of Competitive Programming Library. It aims to be a general template and optimization library for competitive programm
The LiberaPay archive module for the SeanPM life archive project.
By: Top README.md Read this article in a different language Sorted by: A-Z Sorting options unavailable ( af Afrikaans Afrikaans | sq Shqiptare Albania
A feed generator. Currently supports generating RSS feeds from Google, Bing, and Yahoo news.
A feed generator. Currently supports generating RSS feeds from Google, Bing, and Yahoo news.
Simple but maybe too simple config management through python data classes. We use it for machine learning.
π©ββοΈ Coqpit Simple, light-weight and no dependency config handling through python data classes with to/from JSON serialization/deserialization. Curre
Hotpile: High Order Turing Machine Language Compiler
Hotpile: High Order Turing Machine Language Compiler Build and Run Requirements: Python 3.6+, bison, flex, and GCC installed. Needs to be run under UN
Anonymous Dark Web Tool
Anonymous Dark Web Tool v1.0 Features Anonymous Mode Darkweb Search Engines Check Onion Url/s Scanning Host/IP Keep eyes on v2.0 soon. Requirement Deb
Declarative and extensible library for configuration & code separation
ClassyConf ClassyConf is the configuration architecture solution for perfectionists with deadlines. It provides a declarative way to define settings f
It was created to conveniently respond to events such as donation, follow, and hosting using the Alert Box provided by twip to streamers
This library is not an official library of twip. It was created to conveniently respond to events such as donation, follow, and hosting using the Alert Box provided by twip to streamers.
Open Source Repository for CFD Solvers
Background and Validation This wiki is built in Notion. Here are all the tips you need to contribute. General Background Flow over a cylinder The proj
The official FOSSCOMM 2021 CTF by [email protected]
FOSSCOMM 2021 CTF Table of Contents General Info FAQ General Info Purpose: This CTF is a collaboration between the FOSSCOMM conference and the 2 Nov 14, 2021
Nag0mi ctf problem 2021 writeup
Nag0mi ctf problem 2021 writeup
An application for automation of the mining function in the game Alienworlds.IO
alienautomation A Python script made to automate the tidious job of mining on AlienWorlds This script: Automatically opens the browser Automatically l
Semantic Data Management - Property Graphs π
SDM - Lab 1 @ UPC π¨π»βπ» Table of contents Introduction Property Graph Dataset 1. Introduction This repo is all about what we have done in SDM lab 1
Python Service for MISP Feed Management
Python Service for MISP Feed Management This set of scripts is designed to offer better reliability and more control over the fetching of feeds into M
Kunai Shitty Raider Leaked LMFAO
Kunai-Raider-Leaked Kunai Shitty Raider Leaked LMFA
A python package for batch import of resume attachments to be parsed in HrFlow.
HrFlow Importer Description A python package for batch import of resume attachments to be parsed in HrFlow. hrflow-importer is an open-source project
A basic DIY-project made using Python and MySQL
Banking-Using-Python-MySQL This is a basic DIY-project made using Python and MySQL. Pre-Requisite needed:-- MySQL command Line:- creating a database