Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Overview

Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch


WebsiteInstallationMain goalslatest Docsstable DocsCommunityGrid AILicence

PyPI Status PyPI Status Build Status codecov CodeFactor

Documentation Status Slack Discourse status license


Continuous Integration

CI testing
System / PyTorch ver. 1.6 (min. req.) 1.8 (latest)
Linux py3.{6,8} CI full testing CI full testing
OSX py3.{6,8} CI full testing CI full testing
Windows py3.7* CI base testing CI base testing
  • * testing just the package itself, we skip full test suite - excluding tests folder

Install

View install

Simple installation from PyPI

pip install lightning-bolts

Install bleeding-edge (no guarantees)

pip install git+https://github.com/PytorchLightning/[email protected] --upgrade

In case you want to have full experience you can install all optional packages at once

pip install lightning-bolts["extra"]

What is Bolts

Bolts is a Deep learning research and production toolbox of:

  • SOTA pretrained models.
  • Model components.
  • Callbacks.
  • Losses.
  • Datasets.

Main Goals of Bolts

The main goal of Bolts is to enable rapid model idea iteration.

Example 1: Finetuning on data

from pl_bolts.models.self_supervised import SimCLR
from pl_bolts.models.self_supervised.simclr.transforms import SimCLRTrainDataTransform, SimCLREvalDataTransform
import pytorch_lightning as pl

# data
train_data = DataLoader(MyDataset(transforms=SimCLRTrainDataTransform(input_height=32)))
val_data = DataLoader(MyDataset(transforms=SimCLREvalDataTransform(input_height=32)))

# model
weight_path = 'https://pl-bolts-weights.s3.us-east-2.amazonaws.com/simclr/bolts_simclr_imagenet/simclr_imagenet.ckpt'
simclr = SimCLR.load_from_checkpoint(weight_path, strict=False)

simclr.freeze()

# finetune

Example 2: Subclass and ideate

from pl_bolts.models import ImageGPT
from pl_bolts.models.self_supervised import SimCLR

class VideoGPT(ImageGPT):

    def training_step(self, batch, batch_idx):
        x, y = batch
        x = _shape_input(x)

        logits = self.gpt(x)
        simclr_features = self.simclr(x)

        # -----------------
        # do something new with GPT logits + simclr_features
        # -----------------

        loss = self.criterion(logits.view(-1, logits.size(-1)), x.view(-1).long())

        logs = {"loss": loss}
        return {"loss": loss, "log": logs}

Who is Bolts for?

  • Corporate production teams
  • Professional researchers
  • Ph.D. students
  • Linear + Logistic regression heroes

I don't need deep learning

Great! We have LinearRegression and LogisticRegression implementations with numpy and sklearn bridges for datasets! But our implementations work on multiple GPUs, TPUs and scale dramatically...

Check out our Linear Regression on TPU demo

from pl_bolts.models.regression import LinearRegression
from pl_bolts.datamodules import SklearnDataModule
from sklearn.datasets import load_diabetes
import pytorch_lightning as pl

# sklearn dataset
X, y = load_diabetes(return_X_y=True)
loaders = SklearnDataModule(X, y)

model = LinearRegression(input_dim=13)

# try with gpus=4!
# trainer = pl.Trainer(gpus=4)
trainer = pl.Trainer()
trainer.fit(model, train_dataloader=loaders.train_dataloader(), val_dataloaders=loaders.val_dataloader())
trainer.test(test_dataloaders=loaders.test_dataloader())

Is this another model zoo?

No!

Bolts is unique because models are implemented using PyTorch Lightning and structured so that they can be easily subclassed and iterated on.

For example, you can override the elbo loss of a VAE, or the generator_step of a GAN to quickly try out a new idea. The best part is that all the models are benchmarked so you won't waste time trying to "reproduce" or find the bugs with your implementation.

Team

Bolts is supported by the PyTorch Lightning team and the PyTorch Lightning community!


Licence

Please observe the Apache 2.0 license that is listed in this repository. In addition the Lightning framework is Patent Pending.

Citation

To cite bolts use:

@article{falcon2020framework,
  title={A Framework For Contrastive Self-Supervised Learning And Designing A New Approach},
  author={Falcon, William and Cho, Kyunghyun},
  journal={arXiv preprint arXiv:2009.00104},
  year={2020}
}

To cite other contributed models or modules, please cite the authors directly (if they don't have bibtex, ping the authors on a GH issue)

Issues
  • Add RetinaNet Object detection with Backbones

    Add RetinaNet Object detection with Backbones

    What does this PR do?

    Fixes #391

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together?
    • [x] Did you make sure to update the documentation with your changes?
    • [x] Did you write any new necessary tests? [not needed for typos/docs]
    • [x] Did you verify new and existing tests pass locally with your changes?
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [x] Is this pull request ready for review?

    Did you have fun?

    I think yes :stuck_out_tongue:

    0:] Ready-To-Go model 
    opened by oke-aditya 45
  • Add YOLO object detection model

    Add YOLO object detection model

    What does this PR do?

    This PR adds the YOLO object detection model. The implementation is based on the YOLOv3 and YOLOv4 Darknet implementations, although it doesn't include all the features of YOLOv4. Detection seems to work with weights that have been trained using the Darknet implementation, so the network architecture should be more or less identical. The network architecture is read from a configuration file in the same format as in the Darknet implementation. It supports loading weights from a Darknet model file too, if you don't want to start training from a randomly initialized model.

    Fixes #22

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together?
    • [x] Did you make sure to update the documentation with your changes?
    • [x] Did you write any new necessary tests? [not needed for typos/docs]
    • [x] Did you verify new and existing tests pass locally with your changes?
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [x] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    enhancement model datamodule 
    opened by senarvi 36
  • Add SRGAN and datamodules for super resolution

    Add SRGAN and datamodules for super resolution

    What does this PR do?

    Adds a SRGAN implementation to bolts as proposed in #412.

    Closes #412

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
    • [x] Did you make sure to update the documentation with your changes?
    • [x] Did you write any new necessary tests?
    • [x] Did you verify new and existing tests pass locally with your changes?
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG?
    • [x] Add train logs and example images

    PR review

    • [x] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    0:] Ready-To-Go model datamodule 
    opened by chris-clem 31
  • Adding types to some of datamodules

    Adding types to some of datamodules

    What does this PR do?

    Adding types to pl_bolts.datamodules.

    related to #434

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
    • [ ] Did you make sure to update the documentation with your changes?
    • [ ] Did you write any new necessary tests?
    • [ ] Did you verify new and existing tests pass locally with your changes?
    • [ ] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [ ] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    Priority datamodule refactoring 
    opened by briankosw 25
  • Add DCGAN module

    Add DCGAN module

    What does this PR do?

    As proposed in #401, this PR adds a DCGAN implementation closely following the one in PyTorch's examples (https://github.com/pytorch/examples/blob/master/dcgan/main.py).

    Fixes #401

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
    • [x] Did you make sure to update the documentation with your changes?
    • [x] Did you write any new necessary tests?
    • [x] Did you verify new and existing tests pass locally with your changes?
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [x] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    enhancement 0:] Ready-To-Go model 
    opened by chris-clem 24
  • Add EMNISTDataModule

    Add EMNISTDataModule

    What does this PR do?

    Closes #672, #676 and #685.

    A summary of changes and modifications :star: :fire: [CLICK TO EXPAND]

    • File Added:

      • [x] pl_bolts/datasets/emnist_dataset.py :green_circle:
      • [x] Contents:
        • [x] EMNIST_METADATA
        • [x] EMNIST dataset
        • [x] BinaryEMNIST dataset Need New PR or add to #672 :warning:
    • File Added:

      • [x] pl_bolts/datamodules/emnist_dataset.py :green_circle:
      • [x] Contents:
        • [x] EMNISTDataModule
        • [x] BinaryEMNISTDataModule Need New PR or add to #672 :warning:
    • Files Modified

      • Package: pl_bolts

        • [x] pl_bolts/datasets/__init__.py :green_circle:
        • [x] pl_bolts/datamodules/__init__.py :green_circle:
      • Tests:

        • For datamodules:
          • [x] tests/datamodules/test_imports.py :green_circle:
          • [x] tests/datamodules/test_datamodules.py WIP :orange_circle:

    Adding BinaryEMNIST and BinaryEMNISTDataModule was logical, looking at how MNIST and BinaryMNIST (dataset and datamodules) were implemented.

    About the dataset

    image source: https://arxiv.org/pdf/1702.05373.pdf [Table-I]

    image source: https://arxiv.org/pdf/1702.05373.pdf [Table-II]

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements) #672
    • [x] Did you read the contributor guideline, Pull Request section? Y :green_circle:
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together? Y :green_circle:
    • [x] Did you make sure to update the documentation with your changes? Y :green_circle:
    • [x] Did you write any new necessary tests? [not needed for typos/docs] Y :green_circle:
    • [x] Did you verify new and existing tests pass locally with your changes? Y :green_circle:
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG? Y :green_circle:

    PR review

    • [x] Is this pull request ready for review? (if not, please submit in draft mode) READY :green_circle:

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    0:] Ready-To-Go datamodule 
    opened by sugatoray 19
  • Implemented GIoU

    Implemented GIoU

    What does this PR do?

    Implements Generalized Intersection over Union as mentioned in #251

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
    • [x] Did you make sure to update the documentation with your changes?
    • [x] Did you write any new necessary tests?
    • [x] Did you verify new and existing tests pass locally with your changes?
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [x] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    enhancement 
    opened by briankosw 19
  • ci: Fix dataset downloading errors

    ci: Fix dataset downloading errors

    What does this PR do?

    As pointed out in https://github.com/PyTorchLightning/pytorch-lightning-bolts/pull/377#issuecomment-730193148 by @Borda, the tests try to download datasets, which sometimes fail with the following error:

    UNEXPECTED EXCEPTION: RuntimeError('Failed download from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz')

    Description of the changes

    1. ~It seems that those failing tests are often doctest, so this PR simply removes the doctest from ci_test-full.yml as we still have doctest in ci_test-base.yml.~ ~https://github.com/PyTorchLightning/pytorch-lightning-bolts/blob/b8ac85154465956b06fd1005b21b071af5493f11/.github/workflows/ci_test-full.yml#L86~ ~https://github.com/PyTorchLightning/pytorch-lightning-bolts/blob/b8ac85154465956b06fd1005b21b071af5493f11/.github/workflows/ci_test-base.yml#L69~
    2. ~This PR also includes minor changes in some tests using LitMNIST to utilize dataset caching since they currently download and store MNIST datasets in ./ instead of in ./datasets/ (datadir fixture).~ See #414.

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together? Otherwise, we ask you to create a separate PR for every change.
    • [x] Did you make sure to update the documentation with your changes?
    • [ ] Did you write any new necessary tests?
    • [ ] Did you verify new and existing tests pass locally with your changes?
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [ ] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    bug / fix tests / CI datamodule 
    opened by akihironitta 17
  • Adds Backbones to FRCNN Take 2

    Adds Backbones to FRCNN Take 2

    What does this PR do?

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together?
    • [x] Did you make sure to update the documentation with your changes?
    • [x] Did you write any new necessary tests? [not needed for typos/docs]
    • [x] Did you verify new and existing tests pass locally with your changes?
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [x] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    Redo #382 . Closes #340

    0:] Ready-To-Go model 
    opened by oke-aditya 16
  • Add logger for Azure Machine Learning

    Add logger for Azure Machine Learning

    Before submitting

    • [x] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure to update the docs?
    • [x] Did you write any new necessary tests?

    What does this PR do?

    This pull request adds an Azure Machine Learning logger to PyTorch Lightning Bolts. Since AML is integrated with MLFlow for experiment tracking (link), this logger subclasses the MLFlow logger.

    It will enable users to track their metrics in the Azure ML UI by writing code like:

    from azureml.core import Run
    from pytorch_lightning.loggers import AzureMlLogger
    
    # This is optional
    run = Run.get_context()
    azureml_logger = AzureMlLogger(run)
    
    trainer = Trainer(logger=azureml_logger)
    

    This pull request closes #180 .

    PR review

    Everyone is welcome to comment and review!

    Did you have fun?

    Yes!

    enhancement logger has conflicts 
    opened by dkmiller 16
  • Accuracy metric does not work on Model predictions.

    Accuracy metric does not work on Model predictions.

    🐛 Bug

    Error when using pytorch_lightning.metrics.Accuracy on logits, even though that is exactly the use-case stated in the docs.

    To Reproduce

    Steps to reproduce the behavior:

    Using the accuracy metric on either the training or validation step with the expected logit inputs ((64 x 10), 10 = # of classes) and target vector (64) throws the error ValueError: The `preds` should be probabilities, but values were detected outside of [0,1] range. similar to #551.

    Code sample

    class ToyModel(pl.LightningModule):
      def __init__(self, batch_size, max_epochs = 10, learning_rate=0.01, num_classes=10, noise=0.6, alpha=0.1, beta=1):
        super(ToyModel, self).__init__()
    
        # Model
        self.conv1 = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, stride=1, padding=1)
        self.conv1_bn = nn.BatchNorm2d(64)
        
        self.conv2 = nn.Conv2d(64, 128, 3, padding=1)
        self.conv2_bn = nn.BatchNorm2d(128) # pooling changes in_channel for next layer?
    
        self.conv3 = nn.Conv2d(128, 196, 3, padding=1)
        self.conv3_bn = nn.BatchNorm2d(196)
    
        self.fc1 = nn.Linear(in_features=3136, out_features=256)
        self.fc1_bn = nn.BatchNorm1d(256)
    
        self.fc2 = nn.Linear(256, num_classes)
    
        # Parameters
        self.alpha = alpha
        self.beta = beta
        self.noise = noise
    
        self.max_epochs = max_epochs
        self.learning_rate= learning_rate
        self.batch_size = batch_size
    
        # Accuracy Metric
        self.train_acc = pl.metrics.Accuracy()
    
      def loss(self, softmax_pred, target):
        return self.alpha * nn.CrossEntropyLoss()(softmax_pred, target) + self.beta * nn.CrossEntropyLoss()(softmax_pred, target)
    
    
      def forward(self, x):
        x = self.conv1(x)
        x = self.conv1_bn(x)
        x = F.relu(x)
        x = F.max_pool2d(x, kernel_size=2, stride=2)
        
        x = self.conv2(x)
        x = self.conv2_bn(x)
        x = F.relu(x)
        x = F.max_pool2d(x, kernel_size=2, stride=2)
    
        x = self.conv3(x)
        x = self.conv3_bn(x)
        x = F.relu(x)
        x = F.max_pool2d(x, kernel_size=2, stride=2)
        
        x = torch.flatten(x, start_dim=1)
        x = self.fc1(x)
        x = self.fc1_bn(x)
        x = F.relu(x)
    
        return self.fc2(x) 
        
      def training_step(self, batch, batch_idx):
        inputs, targets = batch
        predictions = self(inputs)
        loss = self.loss(predictions, targets)
        self.log("train_acc_step", self.train_acc(predictions, targets))
        return loss
    
      def training_epoch_end(self, outs):
        # log epoch metric
        self.log('train_acc_epoch', self.train_acc.compute())
    

    Passing torch.tensor(64, 3, 32, 32) as training data should trigger the error

    Expected behavior

    self.train_acc(predictions, targets) should compute the accuracy

    Environment

    I used Google Colab

    • PyTorch Version (e.g., 1.0): 1.8.0
    • OS (e.g., Linux): Linux
    • How you installed PyTorch (conda, pip, source): pip
    • Python version: 3.8
    • CUDA/cuDNN version: 11.0

    Additional context

    Workaround: I followed the advice in #551 and reinstalled pytorch lightning to 1.1.8.

    Since this Lightning version triggers #6210 in Colab, I also had to reinstall a different version of torch:

    !pip install torchtext==0.8.0 torch==1.7.1+cu110 torchvision==0.8.2+cu110 torchaudio==0.7.2 -f https://download.pytorch.org/whl/torch_stable.html -q
    !pip install pytorch-lightning==1.1.8 -q
    
    bug / fix help wanted 
    opened by ajsanjoaquin 15
  • CI: Try reusable workflows

    CI: Try reusable workflows

    What does this PR do?

    Just testing out https://github.com/PyTorchLightning/pytorch-lightning/pull/13286.

    Before submitting

    • [ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
    • [ ] Did you read the contributor guideline, Pull Request section?
    • [ ] Did you make sure your PR does only one thing, instead of bundling different changes together?
    • [ ] Did you make sure to update the documentation with your changes?
    • [ ] Did you write any new necessary tests? [not needed for typos/docs]
    • [ ] Did you verify new and existing tests pass locally with your changes?
    • [ ] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [ ] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    opened by akihironitta 0
  • Some datamodules use mnist_transforms variable name in code and docs

    Some datamodules use mnist_transforms variable name in code and docs

    Description

    CIFAR10, TinyCIFAR10DataModule, SSLImagenet, STL10 data modules use mnist_transforms variable name in documentation. When I was reading the docs for CIFAR10 that caused a little confusion. When I was inspecting the code, I noticed that for SSLImageNetDataModule, the variable used to store transformations is also named mnist_transforms. I believe a typo was introduced by copy-paste.

    Files with the typo:

    • pl_bolts\datamodules\cifar10_datamodule.py
    • pl_bolts\datamodules\ssl_imagenet_datamodule.py
    • pl_bolts\datamodules\stl10_datamodule.py

    How to fix

    The most straightforward fix would be updating the docs and updating the variable's name to transforms. We are already in the class that names the dataset, so there is no need to prefix variables. That would also help to avoid similar errors in the future.

    documentation 
    opened by EszKnop 1
  • Call for core contributors 🧙

    Call for core contributors 🧙

    🚀 Feature

    First, we are very happy about all your contribution that the community-made so far! Unfortunately, we are getting a bit short :( Second, we would like to re-ignite this project/repository and get it back on track with the latest research and PL API! Third, as part o the challenge, we are going to rethink the structure and integration process to be up to date and smooth as possible (also see complementary issue #741)

    Motivation

    We want to form a new contributor's team which would be willing to take (participate) this challenge of re-igniting this project in the best Lightning spirit!

    Pitch

    Become a key contributor, collaborate with the best, learn and practice what you love and help us make the Lightning community an even better place!

    Alternatives

    Ping @Borda on slack to chat more...

    Additional context

    Note that to be part of Bolt's core is not the same group as being a Core contributor of the main PL, but it will set you on a promising track to become PL core later on...

    enhancement help wanted discussion 
    opened by Borda 9
  • SwAV example not working

    SwAV example not working

    Hello, I was trying to get the lightning_bolts SwAV example running. unfortunately the example code does not work.

    
    import pytorch_lightning as pl
    from pl_bolts.models.self_supervised import SwAV
    from pl_bolts.datamodules import STL10DataModule
    from pl_bolts.models.self_supervised.swav.transforms import (
        SwAVTrainDataTransform, SwAVEvalDataTransform
    )
    from pl_bolts.transforms.dataset_normalizations import stl10_normalization
    
    # data
    batch_size = 128
    dm = STL10DataModule(data_dir='.', batch_size=batch_size)
    dm.train_dataloader = dm.train_dataloader_mixed
    dm.val_dataloader = dm.val_dataloader_mixed
    
    dm.train_transforms = SwAVTrainDataTransform(
        normalize=stl10_normalization()
    )
    
    dm.val_transforms = SwAVEvalDataTransform(
        normalize=stl10_normalization()
    )
    
    # model
    model = SwAV(
        gpus=1,
        num_samples=dm.num_unlabeled_samples,
        dataset='stl10',
        batch_size=batch_size
    )
    
    # fit
    trainer = pl.Trainer(precision=16)
    trainer.fit(model)
    

    the fit function is missing the data module. It should be: trainer.fit(model, datamodule=dm)

    But even then I get the following error message: RuntimeError: The size of tensor a (128) must match the size of tensor b (0) at non-singleton dimension 0

    How can i fix that? THX

    won't fix 
    opened by sirtris 2
  • Add features to the YOLO model from the latest YOLO variants

    Add features to the YOLO model from the latest YOLO variants

    What does this PR do?

    The YOLO model has been largely refactored, so that it's easy to incorporate new features from different YOLO variants, such as different algorithms for assigning targets to anchors. It also supports defining the network structure as a PyTorch module, in addition to loading a Darknet configuration file. The implementation now supports the most important features of YOLOv3, YOLOv4, YOLOv5, Scaled-YOLOv4, and YOLOX.

    • Supports several new algorithms for matching targets to anchors.
    • Added support for DIoU and CIoU losses.
    • By default the target for confidence of a detection that has been assigned a target is 1.0. This pull request adds a support for using the overlap between the target and the predicted box instead.
    • The code is refactored from one big class into model, anchor matching, and loss function classes.
    • Target class labels may be specified as a matrix of class probabilities, allowing multiple classes per object.
    • Automatic padding in convolutional and max pooling layers works now in every case.
    • Weight decay is applied only to convolutional layer weights.
    • Command line interface is now using LightningCLI.
    • Network architectures can be written as PyTorch modules. YOLOv4, YOLOv5, and YOLOX architectures are included.
    • Calculates MAP metric using TorchMetrics.
    • Added complete type hints for static type checking.

    Fixes #816

    Before submitting

    • [ ] Was this discussed/approved via a Github issue? (no need for typos and docs improvements) - issue for discussion: #816
    • [x] Did you read the contributor guideline, Pull Request section?
    • [x] Did you make sure your PR does only one thing, instead of bundling different changes together?
    • [x] Did you make sure to update the documentation with your changes?
    • [x] Did you write any new necessary tests? [not needed for typos/docs]
    • [x] Did you verify new and existing tests pass locally with your changes?
    • [x] If you made a notable change (that affects users), did you update the CHANGELOG?

    PR review

    • [x] Is this pull request ready for review? (if not, please submit in draft mode)

    Anyone in the community is free to review the PR once the tests have passed. If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

    Did you have fun?

    Make sure you had fun coding 🙃

    model 
    opened by senarvi 0
  • Upgrade the YOLO model with features from new variants

    Upgrade the YOLO model with features from new variants

    🚀 Feature

    Several new variants of the YOLO model have been proposed in the past couple of years. The current implementation in PyTorch Lightning Bolts is based on the original Darknet implementation. I have updated it with features from the new PyTorch based variants and created a pull request.

    Motivation

    The code has been refactored, allowing easy variation of the network architecture, target matching algorithm, or loss function. Some important changes that have been proposed to these algorithms are implemented in this pull request in a modular way. This is the only implementation that combines features from the latest variants, including YOLOv5 and YOLOX, as well as allows loading Darknet based models.

    You can find my pull request here: https://github.com/PyTorchLightning/lightning-bolts/pull/817

    enhancement help wanted 
    opened by senarvi 3
Releases(0.5.0)
  • 0.5.0(Dec 20, 2021)

    [0.5.0] - 2021-12-20

    Added

    • Added YOLO model (#552)
    • Added SRGAN, SRImageLoggerCallback, TVTDataModule, SRCelebA, SRMNIST, SRSTL10 (#466)
    • Added nn.Module support for FasterRCNN backbone (#661)
    • Added RetinaNet with torchvision backbones (#529)
    • Added Python 3.9 support (#786)

    Changed

    • VAE now uses deterministic KL divergence during training, previously estimated KL divergence by random sampling (#760)

    Removed

    • Removed PyTorch 1.6 support (#786)
    • Removed Python 3.6 support (#785)

    Fixed

    • Fixed doctest fails with ImportError: cannot import name 'Env' from 'gym' (#751)
    • Fixed MoCo v2 missing Cosine Annealing learning scheduler (#757)

    Contributors

    @abhayraw1 @akihironitta @chris-clem @hoangtnm @nmichlo @oke-aditya @Programmer-RD-AI @senarvi

    If we forgot someone due to not matching commit email with GitHub account, let us know :]

    Source code(tar.gz)
    Source code(zip)
  • 0.4.0(Sep 9, 2021)

    [0.4.0] - 2021-09-09

    Added

    • Added Soft Actor Critic (SAC) Model (#627)
    • Added EMNISTDataModule, BinaryEMNISTDataModule, and BinaryEMNIST dataset (#676)
    • Added Advantage Actor-Critic (A2C) Model (#598)
    • Added Torch ORT Callback (#720)
    • Added SparseML Callback (#724)

    Changed

    • Changed the default values pin_memory=False, shuffle=False and num_workers=16 to pin_memory=True, shuffle=True and num_workers=0 of datamodules (#701)
    • Supporting deprecated attribute usage (#699)

    Fixed

    • Fixed ImageNet val loader to use val transform instead of train transform (#713)
    • Fixed the MNIST download giving HTTP 404 with torchvision>=0.9.1 (#674)
    • Removed momentum updating from val step and add separate val queue (#631)
    • Fixed moving the queue to GPU when resuming checkpoint for SwAV model (#684)
    • Fixed FP16 support with vision GPT model (#694)
    • Removing bias from linear model regularisation (#669)
    • Fixed CPC module issue (#680)
    Source code(tar.gz)
    Source code(zip)
  • 0.3.4(Jun 17, 2021)

    [0.3.4] - 2021-06-17

    Changed

    • Replaced load_boston with load_diabetes in the docs and tests (#629)
    • Added base encoder and MLP dimension arguments to BYOL constructor (#637)

    Fixed

    • Fixed the MNIST download giving HTTP 503 (#633)
    • Fixed type annotation of ExperienceSource.__iter__ (#645)
    • Fixed pretrained_urls on Windows (#652)
    • Fixed logistic regression (#655, #664)
    • Fixed double softmax in SSLEvaluator (#663)
    Source code(tar.gz)
    Source code(zip)
  • 0.3.3(Apr 17, 2021)

    [0.3.3] - 2021-04-17

    Changed

    • Suppressed missing package warnings, conditioned by WARN_MISSING_PACKAGE="1" (#617)
    • Updated all scripts to LARS (#613)

    Fixed

    • Add missing dataclass requirements (#618)
    Source code(tar.gz)
    Source code(zip)
  • 0.3.2(Mar 20, 2021)

    [0.3.2] - 2021-03-20

    Changed

    • Renamed SSL modules: CPCV2 >> CPC_v2 and MocoV2 >> Moco_v2 (#585)
    • Refactored setup.py to be typing friendly (#601)
    Source code(tar.gz)
    Source code(zip)
  • 0.3.1(Mar 9, 2021)

    [0.3.1] - 2021-03-09

    Added

    • Added Pix2Pix model (#533)

    Changed

    • Moved vision models (GPT2, ImageGPT, SemSegment, UNet) to pl_bolts.models.vision (#561)

    Fixed

    • Fixed BYOL moving average update (#574)
    • Fixed custom gamma in rl (#550)
    • Fixed PyTorch 1.8 compatibility issue (#580, #579)
    • Fixed handling batchnorms in BatchGradientVerification [#569)
    • Corrected num_rows calculation in LatentDimInterpolator callback (#573)

    Contributors

    @akihironitta, @aniketmaurya, @BartekRoszak, @FlorianMF, @indigoviolet, @kaushikb11, @mxksowie, @wjn0

    If we forgot someone due to not matching commit email with GitHub account, let us know :]

    Source code(tar.gz)
    Source code(zip)
  • 0.3.0(Jan 20, 2021)

    Detail chnages

    Added

    • Added input_channels argument to UNet (#297)
    • Added SwAV (#239, #348, #323)
    • Added data monitor callbacks ModuleDataMonitor and TrainingDataMonitor (#285)
    • Added DCGAN module (#403)
    • Added VisionDataModule as parent class for BinaryMNISTDataModule, CIFAR10DataModule, FashionMNISTDataModule, and MNISTDataModule (#400)
    • Added GIoU loss (#347)
    • Added IoU loss (#469)
    • Added semantic segmentation model SemSegment with UNet backend (#259)
    • Added option to normalize latent interpolation images (#438)
    • Added flags to datamodules (#388)
    • Added metric GIoU (#347)
    • Added Intersection over Union Metric/Loss (#469)
    • Added SimSiam model (#407)
    • Added gradient verification callback (#465)
    • Added Backbones to FRCNN (#475)

    Changed

    • Decoupled datamodules from models (#332, #270)
    • Set PyTorch Lightning 1.0 as the minimum requirement (#274)
    • Moved pl_bolts.callbacks.self_supervised.BYOLMAWeightUpdate to pl_bolts.callbacks.byol_updates.BYOLMAWeightUpdate (#288)
    • Moved pl_bolts.callbacks.self_supervised.SSLOnlineEvaluator to pl_bolts.callbacks.ssl_online.SSLOnlineEvaluator (#288)
    • Moved pl_bolts.datamodules.*_dataset to pl_bolts.datasets.*_dataset (#275)
    • Ensured sync across val/test step when using DDP (#371)
    • Refactored CLI arguments of models (#394)
    • Upgraded DQN to use .log (#404)
    • Decoupled DataModules from models - CPCV2 (#386)
    • Refactored datamodules/datasets (#338)
    • Refactored Vision DataModules (#400)
    • Refactored pl_bolts.callbacks (#477)
    • Refactored the rest of pl_bolts.models.self_supervised (#481, #479)
    • Update [torchvision.utils.make_grid(https://pytorch.org/docs/stable/torchvision/utils.html#torchvision.utils.make_grid)] kwargs to TensorboardGenerativeModelImageSampler (#494)

    Fixed

    • Fixed duplicate warnings when optional packages are unavailable (#341)
    • Fixed ModuleNotFoundError when importing datamoules (#303)
    • Fixed cyclic imports in pl_bolts.utils.self_suprvised (#350)
    • Fixed VAE loss to use KL term of ELBO (#330)
    • Fixed dataloders of MNISTDataModule to use self.batch_size (#331)
    • Fixed missing outputs in SSL hooks for PyTorch Lightning 1.0 (#277)
    • Fixed stl10 datamodule (#369)
    • Fixes SimCLR transforms (#329)
    • Fixed binary MNIST datamodule (#377)
    • Fixed the end of batch size mismatch (#389)
    • Fixed batch_size parameter for DataModules remaining (#344)
    • Fixed CIFAR num_samples (#432)
    • Fixed DQN run_n_episodes using the wrong environment variable (#525)

    Contributors

    @akihironitta, @ananyahjha93, @annikabrundyn, @awaelchli, @Borda, @briankosw, @chris-clem, @deng-cy, @hecoding, @miccio-dk, @oke-aditya, @SeanNaren, @sid-sundrani, @teddykoker, @zlapp

    If we forgot someone due to not matching commit email with GitHub account, let us know :]

    Source code(tar.gz)
    Source code(zip)
  • 0.2.3(Oct 12, 2020)

    [0.2.3] - 2020-10-12

    Added

    • Enabled PyTorch Lightning 0.10 compatibility (#264)
    • Added dummy datasets (#266)
    • Added KittiDataModule (#248)
    • Added UNet (#247)
    • Added reinforcement learning models, losses and datamodules (#257)
    Source code(tar.gz)
    Source code(zip)
  • 0.2.1(Sep 13, 2020)

    [0.2.1] - 2020-09-13

    Added

    • Added pretrained VAE with resnet encoders and decoders
    • Added pretrained AE with resnet encoders and decoders
    • Added CPC pretrained on CIFAR10 and STL10
    • Verified BYOL implementation

    Changed

    • Dropped all dependencies except PyTorch Lightning and PyTorch
    • Decoupled datamodules from GAN (#206)
    • Modularize AE & VAE (#196)

    Fixed

    • Fixed gym (#221)
    • Fix L1/L2 regularization (#216)
    • Fix max_depth recursion crash in AsynchronousLoader (#191)
    Source code(tar.gz)
    Source code(zip)
  • 0.1.1(Aug 23, 2020)

    [0.1.1] - 2020-08-23

    Added

    • Added Faster RCNN + Pscal VOC DataModule (#157)
    • Added a better lars scheduling LARSWrapper (#162)
    • Added CPC finetuner (#158)
    • Added BinaryMNISTDataModule (#153)
    • Added learning rate scheduler to BYOL (#148)
    • Added Cityscapes DataModule (#136)
    • Added learning rate scheduler LinearWarmupCosineAnnealingLR (#138)
    • Added BYOL (#144)
    • Added ConfusedLogitCallback (#118)
    • Added an asynchronous single GPU dataloader (#1521)

    Fixed

    • Fixed simclr finetuner (#165)
    • Fixed STL10 finetuner (#164)
    • Fixed Image GPT (#108)
    • Fixed unused MNIST transforms in tran/val/test (#109)

    Changed

    • Enhanced train batch function (#107)
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Jan 16, 2021)

    [0.1.0] - 2020-07-02

    Added

    • Added setup and repo structure
    • Added requirements
    • Added docs
    • Added Manifest
    • Added coverage
    • Added MNIST template
    • Added VAE template
    • Added GAN + AE + MNIST
    • Added Linear Regression
    • Added Moco2g
    • Added simclr
    • Added RL module
    • Added Loggers
    • Added Transforms
    • Added Tiny Datasets
    • Added regularization to linear + logistic models
    • Added Linear and Logistic Regression tests
    • Added Image GPT
    • Added Recommenders module

    Changed

    • Device is no longer set in the DQN model init
    • Moved RL loss function to the losses module
    • Moved rl.common.experience to datamodules
    • train_batch function to VPG model to generate batch of data at each step (POC)
    • Experience source no longer gets initialized with a device, instead the device is passed at each step()
    • Refactored ExperienceSource classes to be handle multiple environments.

    Removed

    • Removed N-Step DQN as the latest version of the DQN supports N-Step by setting the n_step arg to n
    • Deprecated common.experience

    Fixed

    • Documentation
    • Doct tests
    • CI pipeline
    • Imports and pkg
    • CPC fixes
    Source code(tar.gz)
    Source code(zip)
Reference implementation of code generation projects from Facebook AI Research. General toolkit to apply machine learning to code, from dataset creation to model training and evaluation. Comes with pretrained models.

This repository is a toolkit to do machine learning for programming languages. It implements tokenization, dataset preprocessing, model training and m

Facebook Research 320 Jun 16, 2022
Template repository for managing machine learning research projects built with PyTorch-Lightning

Tutorial Repository with a minimal example for showing how to deploy training across various compute infrastructure.

Sidd Karamcheti 3 Feb 11, 2022
Awesome Deep Graph Clustering is a collection of SOTA, novel deep graph clustering methods

ADGC: Awesome Deep Graph Clustering ADGC is a collection of state-of-the-art (SOTA), novel deep graph clustering methods (papers, codes and datasets).

yueliu1999 107 Jun 22, 2022
A collection of SOTA Image Classification Models in PyTorch

A collection of SOTA Image Classification Models in PyTorch

sithu3 68 Jun 15, 2022
NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.

NVIDIA Merlin NVIDIA Merlin is an open source library designed to accelerate recommender systems on NVIDIA’s GPUs. It enables data scientists, machine

null 260 Jun 21, 2022
Facebook Research 567 Jun 18, 2022
A general framework for deep learning experiments under PyTorch based on pytorch-lightning

torchx Torchx is a general framework for deep learning experiments under PyTorch based on pytorch-lightning. TODO list gan-like training wrapper text

Yingtian Liu 6 Mar 17, 2022
An Agnostic Computer Vision Framework - Pluggable to any Training Library: Fastai, Pytorch-Lightning with more to come

IceVision is the first agnostic computer vision framework to offer a curated collection with hundreds of high-quality pre-trained models from torchvision, MMLabs, and soon Pytorch Image Models. It orchestrates the end-to-end deep learning workflow allowing to train networks with easy-to-use robust high-performance libraries such as Pytorch-Lightning and Fastai

airctic 705 Jun 15, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page >> coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 45 Apr 20, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page >> coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 45 Apr 20, 2022
A PyTorch Lightning Callback for pushing models to the Hugging Face Hub 🤗⚡️

hf-hub-lightning A callback for pushing lightning models to the Hugging Face Hub. Note: I made this package for myself, mostly...if folks seem to be i

Nathan Raw 21 May 25, 2022
Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Phil Wang 10.5k Jun 20, 2022
Pretrained Pytorch face detection (MTCNN) and recognition (InceptionResnet) models

Face Recognition Using Pytorch Python 3.7 3.6 3.5 Status This is a repository for Inception Resnet (V1) models in pytorch, pretrained on VGGFace2 and

Tim Esler 2.9k Jun 22, 2022
Official PyTorch implementation and pretrained models of the paper Self-Supervised Classification Network

Self-Classifier: Self-Supervised Classification Network Official PyTorch implementation and pretrained models of the paper Self-Supervised Classificat

Elad Amrani 9 May 15, 2022
Base pretrained models and datasets in pytorch (MNIST, SVHN, CIFAR10, CIFAR100, STL10, AlexNet, VGG16, VGG19, ResNet, Inception, SqueezeNet)

This is a playground for pytorch beginners, which contains predefined models on popular dataset. Currently we support mnist, svhn cifar10, cifar100 st

Aaron Chen 2.3k Jun 17, 2022
(ImageNet pretrained models) The official pytorch implemention of the TPAMI paper "Res2Net: A New Multi-scale Backbone Architecture"

Res2Net The official pytorch implemention of the paper "Res2Net: A New Multi-scale Backbone Architecture" Our paper is accepted by IEEE Transactions o

Res2Net Applications 895 Jun 21, 2022
Implementation of Squeezenet in pytorch, pretrained models on Cifar 10 data to come

Pytorch Squeeznet Pytorch implementation of Squeezenet model as described in https://arxiv.org/abs/1602.07360 on cifar-10 Data. The definition of Sque

gaurav pathak 82 May 13, 2022
FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

Detectron is deprecated. Please see detectron2, a ground-up rewrite of Detectron in PyTorch. Detectron Detectron is Facebook AI Research's software sy

Facebook Research 25.2k Jun 20, 2022