A minimal implementation of the IQRM interference flagging algorithm for radio pulsar and transient searches

Related tags

Algorithmsiqrm
Overview

IQRM

License Build Status codecov

A minimal implementation of the IQRM interference flagging algorithm for radio pulsar and transient searches. This module only provides the algorithm that infers a channel mask from some spectral statistic that measures the level of RFI contamination in a time-frequency data block. It should be useful as a reference implementation to developers who wish to integrate IQRM into an existing pipeline / search code.

However, if you wish to clean existing SIGPROC files using IQRM, please use Kaustubh Rajwade's full implementation:
https://gitlab.com/kmrajwade/iqrm_apollo

Citation

If IQRM contributes to a scientific publication, please cite the article (link will be provided soon):
IQRM: real-time adaptive RFI masking for radio transient and pulsar searches

Installation

Clone the repository an run:

make install

Which installs the module in editable mode, which means you can freely edit the code. The module has very few dependencies: numpy, pytest and pytest-cov for unit tests. They will be automatically installed by pip if not present. To run the unit tests and print a coverage report, simply run:

make tests

Usage

The module provides a single function: iqrm_mask, that operates on an array that contains a spectral statistic that captures the level of RFI contamination in each frequency channel of a search-mode data block. Here is a basic example on an artificially generated time-frequency data block:

import numpy as np
from iqrm import iqrm_mask

### Generate a time-frequency block containing Gaussian noise
nsamp = 1024
nchan = 10
data = np.random.normal(size=nsamp*nchan).reshape(nsamp, nchan)

# Add simulated RFI to channels indices 5 and 6
data[:, 5] += 5 * np.sin(np.arange(nsamp))
data[:, 6] += 10 * np.sin(np.arange(nsamp))

# Use per-channel standard deviation as contamination measure
spectral_std = data.std(axis=0)

### Run IQRM
mask, votes = iqrm_mask(spectral_std, radius=2)

# 'mask' is a boolean mask where 'True' denotes a channel index deemed to be contaminated
print(np.where(mask)[0])
# Output:
# [5 6]

# 'votes' is a dictionary of sets; see section 2 of the paper for an explanation of the 'voting' system
for caster in sorted(votes.keys()):
    recipients = votes[caster]
    print(caster, recipients)
# Output:
# 3 {5}
# 4 {5, 6}
# 5 {6}
# 7 {5, 6}
# 8 {6}

# For example '7 {5, 6}' means that from the point of view of channel 7, channels 5 and 6 have an abnormally high level of RFI contamination.
Owner
Vincent Morello
Postdoc in the MeerTRAP group, JBCA, University of Manchester. FRB and pulsar searching, statistics, Python, what else.
Vincent Morello
There are some basic arithmatic in Pattern Recognization and Machine Learning writed in Python in this repository

There are some basic arithmatic in Pattern Recognization and Machine Learning writed in Python in this repository

1 Nov 19, 2021
Fedlearn algorithm toolkit for researchers

Fedlearn algorithm toolkit for researchers

89 Nov 14, 2022
This is an implementation of the QuickHull algorithm in Python. I

QuickHull This is an implementation of the QuickHull algorithm in Python. It randomly generates a set of points and finds the convex hull of this set

Anant Joshi 4 Dec 04, 2022
Infomap is a network clustering algorithm based on the Map equation.

Infomap Infomap is a network clustering algorithm based on the Map equation. For detailed documentation, see mapequation.org/infomap. For a list of re

347 Dec 23, 2022
Python algorithm to determine the optimal elevation threshold of a GNSS receiver, by using a statistical test known as the Brown-Forsynthe test.

Levene and Brown-Forsynthe: Test for variances Application to Global Navigation Satellite Systems (GNSS) Python algorithm to determine the optimal ele

Nicolas Gachancipa 2 Aug 09, 2022
Genetic Algorithm for Robby Robot based on Complexity a Guided Tour by Melanie Mitchell

Robby Robot Genetic Algorithm A Genetic Algorithm based Robby the Robot in Chapter 9 of Melanie Mitchell's book Complexity: A Guided Tour Description

Matthew 2 Dec 01, 2022
Implementation of an ordered dithering algorithm used in computer graphics

Ordered Dithering Project In this project, we use an ordered dithering method to turn an RGB image, first to a gray scale image and then, turn the gra

1 Oct 26, 2021
Distributed Grid Descent: an algorithm for hyperparameter tuning guided by Bayesian inference, designed to run on multiple processes and potentially many machines with no central point of control

Distributed Grid Descent: an algorithm for hyperparameter tuning guided by Bayesian inference, designed to run on multiple processes and potentially many machines with no central point of control.

Martin 1 Jan 01, 2022
An open source algorithm and dataset for finding poop in pictures.

The shitspotter module is where I will be work on the "shitspotter" poop-detection algorithm and dataset. The primary goal of this work is to allow for the creation of a phone app that finds where yo

Jon Crall 29 Nov 29, 2022
Implementation for Evolution of Strategies for Cooperation

Moraliser Implementation for Evolution of Strategies for Cooperation Dependencies You will need a python3 (= 3.8) environment to run the code. Before

1 Dec 21, 2021
This is the code repository for 40 Algorithms Every Programmer Should Know , published by Packt.

40 Algorithms Every Programmer Should Know, published by Packt

Packt 721 Jan 02, 2023
BCI datasets and algorithms

Brainda Welcome! First and foremost, Welcome! Thank you for visiting the Brainda repository which was initially released at this repo and reorganized

52 Jan 04, 2023
A GUI visualization of QuickSort algorithm

QQuickSort A simple GUI visualization of QuickSort algorithm. It only uses PySide6, it does not have any other external dependency. How to run Install

Jaime R. 2 Dec 24, 2021
Policy Gradient Algorithms (One Step Actor Critic & PPO) from scratch using Numpy

Policy Gradient Algorithms From Scratch (NumPy) This repository showcases two policy gradient algorithms (One Step Actor Critic and Proximal Policy Op

1 Jan 17, 2022
Algorithmic virtual trading using the neostox platform

Documentation Neostox doesnt have an API Support, so this is a little selenium code to automate strategies How to use Clone this repository and then m

Abhishek Mittal 3 Jul 20, 2022
Slight modification to one of the Facebook Salina examples, to test the A2C algorithm on financial series.

Facebook Salina - Gym_AnyTrading Slight modification of Facebook Salina Reinforcement Learning - A2C GPU example for financial series. The gym FOREX d

Francesco Bardozzo 5 Mar 14, 2022
Algorithms for calibrating power grid distribution system models

Distribution System Model Calibration Algorithms The code in this library was developed by Sandia National Laboratories under funding provided by the

Sandia National Laboratories 2 Oct 31, 2022
zoofs is a Python library for performing feature selection using an variety of nature inspired wrapper algorithms. The algorithms range from swarm-intelligence to physics based to Evolutionary. It's easy to use ,flexible and powerful tool to reduce your feature size.

zoofs is a Python library for performing feature selection using a variety of nature-inspired wrapper algorithms. The algorithms range from swarm-intelligence to physics-based to Evolutionary. It's e

Jaswinder Singh 168 Dec 30, 2022
This python algorithm creates a simple house floor plan based on a user-provided CSV file.

This python algorithm creates a simple house floor plan based on a user-provided CSV file. The algorithm generates possible router placements and evaluates where a signal will be reached in every roo

Joshua Miller 1 Nov 12, 2021
A* (with 2 heuristic functions), BFS , DFS and DFS iterativeA* (with 2 heuristic functions), BFS , DFS and DFS iterative

Descpritpion This project solves the Taquin game (jeu de taquin) problem using different algorithms : A* (with 2 heuristic functions), BFS , DFS and D

Ayari Ahmed 3 May 09, 2022