BErt-like Neurophysiological Data Representation

Related tags

Data AnalysisBENDR
Overview

BENDR

BErt-like Neurophysiological Data Representation

A picture of Bender from Futurama

This repository contains the source code for reproducing, or extending the BERT-like self-supervision pre-training for EEG data from the article:

BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

To run these scripts, you will need to use the DN3 project. We will try to keep this updated so that it works with the latest DN3 release. If you are just looking for the BENDR model, and don't need to reproduce the article results per se, BENDR will be (or maybe already is if I forgot to update it here) integrated into DN3, in which case I would start there.

Currently, we recommend version 0.2. Feel free to open an issue if you are having any trouble.

More extensive instructions are upcoming, but in essence you will need to either:

a)  Download the TUEG dataset and pre-train new encoder and contextualizer weights, _or_
b)  Use the [pre-trained model weights](https://github.com/SPOClab-ca/BENDR/releases/tag/v0.1-alpha)

Once you have a pre-trained model:

1) Add the paths of the pre-trained weights to configs/downstream.yml
2) Edit paths to local copies of your datasets in configs/downstream_datasets.yml
3) Run downstream.sh

Comments
  • about the loss function

    about the loss function

    Very appreciate for your contribution.i am really interested in the self training in EEG. The only question is about calculating loss function. In your paper, The calculation of the denominator uses cosine similarity between the output of the transformer and the 20 distractors and the input of the transformer. However, in the code, the calculation of the denominator uses cosine similarity between the input of the transformer and the 20 distractors, and the output of the transformer. In other word, the output and the input switch positions. Are both the calculation approaches the same? Or why did you change the calculation approache in the code? Thanks!

    opened by stickOverCarrot 2
  • About deploy downstream.yml and downstream_datasets.yml

    About deploy downstream.yml and downstream_datasets.yml

    Tranks for supplying your code. But when I follow your markdown, I meet some problems image

    This is my project files image

    This is my downstream.yml image

    This is my downstream_datasets.yml image

    opened by YoloEliwa 1
  • Pre-trained weights?

    Pre-trained weights?

    Not an issue per se, but you state the pre-trained weights for your paper are available in this repo, yet I have had a good look around and I haven't found them, nor a means of downloading them. Please can you let me know where I could find them? I'm really keen to try out this exciting architecture you've put together!

    opened by SgtWhiskeyjack 1
  • result_tracking module

    result_tracking module

    There's a reference that's in the module import: downstream.py from result_tracking import ThinkerwiseResultTracker that looks like some type of tracking code for experiments?

    opened by bencten 1
  • dropout should change

    dropout should change

    Iteration: 4%|▍ | 13/330 [00:36<16:00, 3.03s/batches, bac=0.5, Accuracy=0.51, loss=0.695, lr=1.47e-6]D:\Anaconda\envs\LGG\lib\site-packages\torch\nn\functional.py:1338: UserWarning: dropout2d: Received a 3D input to dropout2d and assuming that channel-wise 1D dropout behavior is desired - input is interpreted as shape (N, C, L), where C is the channel dim. This behavior will change in a future release to interpret the input as one without a batch dimension, i.e. shape (C, H, W). To maintain the 1D channel-wise dropout behavior, please switch to using dropout1d instead. warnings.warn("dropout2d: Received a 3D input to dropout2d and assuming that channel-wise "

    opened by zy2021314 0
  • A more detailed explanation

    A more detailed explanation

    We need to use your code for research, may I ask when you can provide detailed explanation, because we have some difficulties in understanding the code without detailed explanation.

    opened by EchizenMike 0
  • preload in downstream.yml

    preload in downstream.yml

    In the "downstream.yml" file, what is the function of the "preload"? What's mean if I specify "preload: True" or "preload: False"?

    Thank you in advance

    opened by frannfuri 0
Releases(v0.1-alpha)
Owner
GitHub Repository
TextDescriptives - A Python library for calculating a large variety of statistics from text

A Python library for calculating a large variety of statistics from text(s) using spaCy v.3 pipeline components and extensions. TextDescriptives can be used to calculate several descriptive statistic

150 Dec 30, 2022
Analyze the Gravitational wave data stored at LIGO/VIRGO observatories

Gravitational-Wave-Analysis This project showcases how to analyze the Gravitational wave data stored at LIGO/VIRGO observatories, using Python program

1 Jan 23, 2022
Show you how to integrate Zeppelin with Airflow

Introduction This repository is to show you how to integrate Zeppelin with Airflow. The philosophy behind the ingtegration is to make the transition f

Jeff Zhang 11 Dec 30, 2022
INF42 - Topological Data Analysis

TDA INF421(Conception et analyse d'algorithmes) Projet : Topological Data Analysis SphereMin Etant donné un nuage des points, ce programme contient de

2 Jan 07, 2022
Single machine, multiple cards training; mix-precision training; DALI data loader.

Template Script Category Description Category script comparison script train.py, loader.py for single-machine-multiple-cards training train_DP.py, tra

2 Jun 27, 2022
MidTerm Project for the Data Analysis FT Bootcamp, Adam Tycner and Florent ZAHOUI

MidTerm Project for the Data Analysis FT Bootcamp, Adam Tycner and Florent ZAHOUI Hallo

Florent Zahoui 1 Feb 07, 2022
Project under the certification "Data Analysis with Python" on FreeCodeCamp

Sea Level Predictor Assignment You will anaylize a dataset of the global average sea level change since 1880. You will use the data to predict the sea

Bhavya Gopal 3 Jan 31, 2022
Used for data processing in machine learning, and help us to construct ML model more easily from scratch

Used for data processing in machine learning, and help us to construct ML model more easily from scratch. Can be used in linear model, logistic regression model, and decision tree.

ShawnWang 0 Jul 05, 2022
DenseClus is a Python module for clustering mixed type data using UMAP and HDBSCAN

DenseClus is a Python module for clustering mixed type data using UMAP and HDBSCAN. Allowing for both categorical and numerical data, DenseClus makes it possible to incorporate all features in cluste

Amazon Web Services - Labs 53 Dec 08, 2022
An orchestration platform for the development, production, and observation of data assets.

Dagster An orchestration platform for the development, production, and observation of data assets. Dagster lets you define jobs in terms of the data f

Dagster 6.2k Jan 08, 2023
Stream-Kafka-ELK-Stack - Weather data streaming using Apache Kafka and Elastic Stack.

Streaming Data Pipeline - Kafka + ELK Stack Streaming weather data using Apache Kafka and Elastic Stack. Data source: https://openweathermap.org/api O

Felipe Demenech Vasconcelos 2 Jan 20, 2022
Senator Trades Monitor

Senator Trades Monitor This monitor will grab the most recent trades by senators and send them as a webhook to discord. Installation To use the monito

Yousaf Cheema 5 Jun 11, 2022
Reading streams of Twitter data, save them to Kafka, then process with Kafka Stream API and Spark Streaming

Using Streaming Twitter Data with Kafka and Spark Reading streams of Twitter data, publishing them to Kafka topic, process message using Kafka Stream

Rustam Zokirov 1 Dec 06, 2021
Generate lookml for views from dbt models

dbt2looker Use dbt2looker to generate Looker view files automatically from dbt models. Features Column descriptions synced to looker Dimension for eac

lightdash 126 Dec 28, 2022
Statistical & Probabilistic Analysis of Store Sales, University Survey, & Manufacturing data

Statistical_Modelling Statistical & Probabilistic Analysis of Store Sales, University Survey, & Manufacturing data Statistical Methods for Decision Ma

Avnika Mehta 1 Jan 27, 2022
sportsdataverse python package

sportsdataverse-py See CHANGELOG.md for details. The goal of sportsdataverse-py is to provide the community with a python package for working with spo

Saiem Gilani 37 Dec 27, 2022
🌍 Create 3d-printable STLs from satellite elevation data 🌏

mapa 🌍 Create 3d-printable STLs from satellite elevation data Installation pip install mapa Usage mapa uses numpy and numba under the hood to crunch

Fabian Gebhart 13 Dec 15, 2022
scikit-survival is a Python module for survival analysis built on top of scikit-learn.

scikit-survival scikit-survival is a Python module for survival analysis built on top of scikit-learn. It allows doing survival analysis while utilizi

Sebastian Pölsterl 876 Jan 04, 2023
Scraping and analysis of leetcode-compensations page.

Leetcode compensations report Scraping and analysis of leetcode-compensations page.

utsav 96 Jan 01, 2023
Advanced Pandas Vault — Utilities, Functions and Snippets (by @firmai).

PandasVault ⁠— Advanced Pandas Functions and Code Snippets The only Pandas utility package you would ever need. It has no exotic external dependencies

Derek Snow 374 Jan 07, 2023