A simplified framework and utilities for PyTorch

Overview

Poutyne Logo

License: LGPL v3 Continuous Integration

Here is Poutyne.

Poutyne is a simplified framework for PyTorch and handles much of the boilerplating code needed to train neural networks.

Use Poutyne to:

  • Train models easily.
  • Use callbacks to save your best model, perform early stopping and much more.

Read the documentation at Poutyne.org.

Poutyne is compatible with the latest version of PyTorch and Python >= 3.6.

Cite

@misc{poutyne,
    author = {Paradis, Fr{\'e}d{\'e}rik and Beauchemin, David and Godbout, Mathieu and Alain, Mathieu and Garneau, Nicolas and Otte, Stefan and Tremblay, Alexis and B{\'e}langer, Marc-Antoine and Laviolette, Fran{\c{c}}ois},
    title  = {{Poutyne: A Simplified Framework for Deep Learning}},
    year   = {2020},
    note   = {\url{https://poutyne.org}}
}

Getting started: few seconds to Poutyne

The core data structure of Poutyne is a Model, a way to train your own PyTorch neural networks.

How Poutyne works is that you create your PyTorch module (neural network) as usual but when comes the time to train it you feed it into the Poutyne Model, which handles all the steps, stats and callbacks, similar to what Keras does.

Here is a simple example:

# Import the Poutyne Model and define a toy dataset
from poutyne import Model
import torch
import torch.nn as nn
import numpy as np

num_features = 20
num_classes = 5
hidden_state_size = 100

num_train_samples = 800
train_x = np.random.randn(num_train_samples, num_features).astype('float32')
train_y = np.random.randint(num_classes, size=num_train_samples).astype('int64')

num_valid_samples = 200
valid_x = np.random.randn(num_valid_samples, num_features).astype('float32')
valid_y = np.random.randint(num_classes, size=num_valid_samples).astype('int64')

num_test_samples = 200
test_x = np.random.randn(num_test_samples, num_features).astype('float32')
test_y = np.random.randint(num_classes, size=num_test_samples).astype('int64')

Select a PyTorch device so that it runs on GPU if you have one:

cuda_device = 0
device = torch.device("cuda:%d" % cuda_device if torch.cuda.is_available() else "cpu")

Create yourself a PyTorch network:

network = nn.Sequential(
    nn.Linear(num_features, hidden_state_size),
    nn.ReLU(),
    nn.Linear(hidden_state_size, num_classes)
)

You can now use Poutyne's model to train your network easily:

model = Model(network, 'sgd', 'cross_entropy',
              batch_metrics=['accuracy'], epoch_metrics=['f1'],
              device=device)
model.fit(
    train_x, train_y,
    validation_data=(valid_x, valid_y),
    epochs=5,
    batch_size=32
)

Since Poutyne is inspired by Keras, one might have notice that this is really similar to some of its functions.

You can evaluate the performances of your network using the evaluate method of Poutyne's model:

loss, (accuracy, f1score) = model.evaluate(test_x, test_y)

Or only predict on new data:

predictions = model.predict(test_x)

See the complete code here. Also, see this for an example for regression that again also uses epoch metrics.

One of the strengths Poutyne are callbacks. They allow you to save checkpoints, log training statistics and more. See this notebook for an introduction to callbacks. In that vein, Poutyne also offers an Experiment class that offers automatic checkpointing, logging and more using callbacks under the hood. Here is an example of usage.

from poutyne import Experiment, TensorDataset
from torch.utils.data import DataLoader

# We need to use dataloaders (i.e. an iterable of batches) with Experiment
train_loader = DataLoader(TensorDataset(train_x, train_y), batch_size=32)
valid_loader = DataLoader(TensorDataset(valid_x, valid_y), batch_size=32)
test_loader = DataLoader(TensorDataset(test_x, test_y), batch_size=32)

# Everything is saved in ./expt/my_classification_network
expt = Experiment('./expt/my_classification_network', network, device=device, optimizer='sgd', task='classif')

expt.train(train_loader, valid_loader, epochs=5)

expt.test(test_loader)

See the complete code here. Also, see this for an example for regression that again also uses epoch metrics.


Installation

Before installing Poutyne, you must have the latest version of PyTorch in your environment.

  • Install the stable version of Poutyne:
pip install poutyne
  • Install the latest development version of Poutyne:
pip install -U git+https://github.com/GRAAL-Research/[email protected]

Learning Material

Blog posts

  • Medium PyTorch post - Presentation of the basics of Poutyne and how it can help you be more efficient when developing neural networks with PyTorch.

Examples

Look at notebook files with full working examples:

or in Google Colab:

Videos


Contributing to Poutyne

We welcome user input, whether it is regarding bugs found in the library or feature propositions ! Make sure to have a look at our contributing guidelines for more details on this matter.


License

Poutyne is LGPLv3 licensed, as found in the LICENSE file.


Why this name, Poutyne?

Poutyne's name comes from poutine, the well-known dish from Quebec. It is usually composed of French fries, squeaky cheese curds and brown gravy. However, in Quebec, poutine also has the meaning of something that is an "ordinary or common subject or activity". Thus, Poutyne will get rid of the ordinary boilerplate code that plain PyTorch training usually entails.

Poutine Yuri Long from Arlington, VA, USA [CC BY 2.0]


Comments
  • [WIP] Adds Dockerfile

    [WIP] Adds Dockerfile

    This PR aims to add a Dockerfile for the poutyne.

    @freud14 while this image gets successfully built, I'd like to further tone down the image size. At the moment it's about 6.35 GB which makes no sense as the base image itself is 2.79 GB. After using dive to inspect the image it turns out the COPY statement from the base image takes up around ~6.2GB.

    I've never encountered this problem before, would be grateful if somebody else pitches in.

    To build the 🐳 image use DOCKER_BUILDKIT=1 docker build -t poutyne .

    opened by SauravMaheshkar 17
  • Introduce

    Introduce "Train Phase Interface"

    This PR is going to introduce a fastai-like "train phase interface" and create the foundation for more complex and fine grained control over the training process.

    The PR addresses and closes #15.

    This PR adds the following

    • Parameter spaces: LinSpace and CosineSpace lazily generate values in the specified domain.
    • The TrainPhase class uses the parameter spaces and specifies how to configure the optimizer in each step. It return a dict of optimizer parameters.
    • The TrainPolicy combines different TrainPhases and updates the optimizer according to the parameters returned by the TrainPolicys.

    TODO

    • [x] first implementation
    • [x] tweak implementation
    • [x] Store the state of the TrainPolicy and make it restartable
    • [x] add "one cycle" policy
    • [x] add "sgdr" policy
    • [x] fully test everything (and switch from pytest back to UnitTest)
      • [x] spaces
      • [x] TrainPhase
      • [x] TrainPolicy
      • [x] integration test
    • [x] docs
      • [x] write docstrings
      • [x] add to sphinx
      • [x] clean up sphinx warnings
    • [x] add references to the papers
    • [x] create a example notebook
    • [x] clean up commit history

    Possible extensions

    • Different optimizers could be set for each phase
    • More policies
    • Set different learning rates for different part of the network
    opened by sotte 15
  • Make logo and README to the project

    Make logo and README to the project

    The logo should be inspired from pytorch's Logo and have a log (like a wood log y'know) in fire. The description of the project in the README should start with something like "Pytoune is a Keras-like interface for the PyTorch framework...." and should have SEO (search engine optimization) in mind. I don't know what it is possible for that in github/markdown. We should also includes lots of examples of what's possible with our framework.

    enhancement good first issue in progress 
    opened by freud14 15
  • Loss function that takes a data dependent 3rd input

    Loss function that takes a data dependent 3rd input

    Is there anyway to implement the functionality as described in this post. https://stackoverflow.com/questions/46464549/keras-custom-loss-function-accessing-current-input-pattern

    Gist: implementing something like code below

        def custom_loss_wrapper(input_tensor):
            def custom_loss(y_true, y_pred):
                return K.binary_crossentropy(y_true, y_pred) + K.mean(input_tensor)
       return custom_loss
    

    This can be called with loss=custom_loss_wrapper(model.input)

    opened by prabhuteja12 12
  • problem about predict

    problem about predict

    due to lack of predict examples, I am unable to use the model.predict correctly, there also exists a bug as: RuntimeError: Expected 4-dimensional input for 4-dimensional weight [256, 1, 1, 3], but got 3-dimensional input of size [31, 2, 132] instead I used :output=model.predict_on_batch(Singledata) and my Singledata is a array of size (31,2,128), float type

    bug 
    opened by azyslzp 11
  • Group all tensorboard metrics in one experiment run

    Group all tensorboard metrics in one experiment run

    Hi,

    I believe current tensorboard callback does not use tensorboard summaries the way they were intended.

    Currently different metrics groups are stored as different runs (left-hand side of tensorboard).

    Those are meant to represent different experiment run that you can compare against themselves. With current implementation this kind of bookkeeping is very difficult.

    This is current implementation:

    old_way

    With current approach is hard to compare different experiment and number of 'run' entries on left is overwhelming.

    This PR makes TensorBoard callback behave as its counterpart in keras. All metrics are stored within one run only. That way you can easily compare different experiments.

    See second screenshot for comparison:

    tensorboard_1

    Now I can easily see how many experiments I have and I can easily compare them against each other. If I want to select some plots subset on the right I can selectively hide/show any chart and/or use regex query. No need to clutter the 'runs' list!

    opened by Ajk4 11
  • [BUG]Failed building wheel for torch

    [BUG]Failed building wheel for torch

    Dear sir: I came across an error when I installed poutyne with "pip install -U git+https://github.com/GRAAL-Research/[email protected]". I think the problem is in my torch.As the Error information said: "ERROR: Failed building wheel for torch". But I have no idea in solving this.So I hope you can provide some solution to me. The overall error feedback is in below. Thanks!

    ERROR: Command errored out with exit status 1: command: 'c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py'"'"'; file='"'"'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-wheel-f_6axmet' cwd: C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch
    Complete output (30 lines): running bdist_wheel running build running build_deps Traceback (most recent call last): File "", line 1, in File "C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py", line 225, in setup(name="torch", version="0.1.2.post2", File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\site-packages\setuptools_init_.py", line 144, in setup return distutils.core.setup(**attrs) File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\core.py", line 148, in setup dist.run_commands() File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 966, in run_commands self.run_command(cmd) File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\site-packages\wheel\bdist_wheel.py", line 223, in run self.run_command('build') File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\command\build.py", line 135, in run self.run_command(cmd_name) File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py", line 51, in run from tools.nnwrap import generate_wrappers as generate_nn_wrappers ModuleNotFoundError: No module named 'tools.nnwrap'

    ERROR: Failed building wheel for torch Running setup.py clean for torch ERROR: Command errored out with exit status 1: command: 'c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py'"'"'; file='"'"'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' clean --all cwd: C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch Complete output (2 lines): running clean error: [Errno 2] No such file or directory: '.gitignore'

    ERROR: Failed cleaning build dir for torch Successfully built Poutyne Failed to build torch Installing collected packages: torch, Poutyne Running setup.py install for torch ... error ERROR: Command errored out with exit status 1: command: 'c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py'"'"'; file='"'"'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\彭 张智computational AI\AppData\Local\Temp\pip-record-e8il4qu8\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\Include\torch' cwd: C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch
    Complete output (23 lines): running install running build_deps Traceback (most recent call last): File "", line 1, in File "C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py", line 225, in setup(name="torch", version="0.1.2.post2", File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\site-packages\setuptools_init_.py", line 144, in setup return distutils.core.setup(**attrs) File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\core.py", line 148, in setup dist.run_commands() File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 966, in run_commands self.run_command(cmd) File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py", line 99, in run self.run_command('build_deps') File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\cmd.py", line 313, in run_command self.distribution.run_command(command) File "c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\lib\distutils\dist.py", line 985, in run_command cmd_obj.run() File "C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py", line 51, in run from tools.nnwrap import generate_wrappers as generate_nn_wrappers ModuleNotFoundError: No module named 'tools.nnwrap' ---------------------------------------- ERROR: Command errored out with exit status 1: 'c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\python.exe' -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py'"'"'; file='"'"'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-install-fgpcp5ez\torch\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\彭张智computational AI\AppData\Local\Temp\pip-record-e8il4qu8\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\彭 张智computational ai\appdata\local\programs\python\python38-32\Include\torch' Check the logs for full command output. WARNING: You are using pip version 20.0.2; however, version 20.2.3 is available. You should consider upgrading via the 'c:\users\彭张智computational ai\appdata\local\programs\python\python38-32\python.exe -m pip install --upgrade pip' command.

    bug 
    opened by pengzhangzhi 10
  • Multi io

    Multi io

    The core modification was to create a new TensorData that would recursively fetch the proper index value and a function that can aggregate batches of output as one big output.

    I disabled one test test_disable_batch_size_warning as I am not sure if it's required anymore. I modified _get_batch_size to account for this new input/output structure, but I am not sure what the original intention was.

    opened by atremblay 9
  • Allow LRScheduler to step at the end of batches

    Allow LRScheduler to step at the end of batches

    TL;DR: some LRSchedulers must be able to update the learning rate (and other parameters) during the batches and not just at the end of an epoch. Should we implement this? What is the best way?

    Long version:

    I would love to see PyToune offer advanced learning rate schedules like 1 cycle policy!

    PyTorch's _LRScheduler have a step() method which set the learning rate. The user is then responsible to call the step() method somewhere in the training loop. This normally happens at the end of each epoch, but it could also happen after each batch.

    PyToune wraps PyTorch's _LRSchedulers via the Callback interface and calls the step() method on_epoch_end. This is great for convenience, but some LRSchedulers have to update the lr after each batch.

    Here are a few options I see:

    • We could update the PyTorchLRSchedulerWrapper and make it configurable when to trigger the step() call, and then use it something like this:

      CosineAnnealingLR(step_on_batch_end=True, ...)
      
    • We could introduce batch versions of all LRSchedulers.

    • I use a simple callback LRSetter that gets a list of learning rates (and momentum values) and sets the lr (and momentum) on each batch end.

    enhancement 
    opened by sotte 9
  • Gist to convert python objects into pytorch tensors

    Gist to convert python objects into pytorch tensors

    Hi,

    I added a gist with a couple of functions to convert a numpy array (or python object such as list) into a pytorch tensor. I couldn't find anything like this in the library except a function going the other way around poutyne.torch_to_numpy.

    These functions are copied from the FastAI library (core module) and modified a bit. If this is useful, I could submit a PR.

    opened by sudarshan85 9
  • Multiple outputs

    Multiple outputs

    Would it be wise to support multiple output? At least a pass through? I have a model that returns two outputs (prediction and attention weights). I had to do a small wrapper around the loss function as such

        def loss_function(y_pred, y_true):
            output, word_attention = y_pred
            return criterion(output, y_true)
    
        def acc(y_pred, y_true, *args):
            y_pred, _ = y_pred
            return metrics.bin_acc(y_pred, y_true, *args)
    

    Which is totally normal considering that this is an uncommon output and there is no way to know how to calculate the loss or the metric in a generic way. Training was working just fine with this. But when it came to predict() it couldn't get through because of

    https://github.com/GRAAL-Research/poutyne/blob/master/poutyne/framework/model.py#L406

    So I experimented a little bit and came up with this function in utils.py

    def _concat(obj):
        if isinstance(obj[0], tuple):
            return tuple([_concat(ele) for ele in zip(*obj)])
        else:
            return np.concatenate(obj)
    

    Then in predict() I call it instead of np.concatenate directly

            pred_y = self.predict_generator(generator)
            return utils._concat(pred_y)
    

    This will work as long as the model returns either a tuple or multiple values at once return (output1, output2) or return output1, output2.

    I don't know if it's a reasonable expectation. All the tests are passing. Since it requires custom loss functions (and metrics), there doesn't seem to be any new tests to add.

    Just wanted your thoughts on this.

    Thanks

    opened by atremblay 8
  • Use FScore (Macro) as monitor metric instead of accuracy for Classification Task [FEATURE]

    Use FScore (Macro) as monitor metric instead of accuracy for Classification Task [FEATURE]

    When training for a classification task, FScore is more relevant than accuracy. After training, when I select the best model, I often get a subpar model, since a bette model with higher FScore may be available, and no checkpoint were log for it

    Either use FScore as default monitor metric for classification or additionnal task such as "unbalanced_classification".

    At the moment I have to manually add the FScore Macro and max as monitor mode. Very easy to fix - I just don't see why someone with a normal classification task would select accuracy over FScore as target metric.

    enhancement 
    opened by jtbai 0
  • Add usage examples

    Add usage examples

    It would be nice to add more examples of usage. For instance, how to use the available callbacks, etc. The examples could be in both the doc and in an "examples" folder.

    enhancement good first issue 
    opened by freud14 3
  • Add a way to add regularizers more easily

    Add a way to add regularizers more easily

    It could be nice to have some kind of way to add regularization like it is possible in Keras with the add_loss function. Maybe adding some optional list of losses in the module that would be summed up by the Module class?

    enhancement 
    opened by freud14 1
  • Add a wrapper for initializers

    Add a wrapper for initializers

    This should includes wrappers for pytorch modules, which do not support initializers in their constructors.

    Furthermore, there should be some kind of way to take a given pytorch module and initialize it with default and user-chosen intializers depending on the type of the modules inside the given module. For instance, say I have some module A which contains other modules and some of them are of type Linear. The functionality should take the module A as a parameters and initialize the Linear modules in some default or non-default way.

    enhancement 
    opened by ngarneau 0
Releases(v1.14)
  • v1.14(Dec 2, 2022)

    • Update examples using classification metrics from torchmetrics to add the now required task argument.
    • Fix the no LR scheduler bug when using PyTorch 2.0.
    Source code(tar.gz)
    Source code(zip)
  • v1.13(Nov 5, 2022)

    Breaking changes:

    • The deprecated torch_metrics keyword argument has been removed. Users should use the batch_metrics or epoch_metrics keyword argument for torchmetrics' metrics.
    • The deprecated EpochMetric class has been removed. Users should implement the Metric class instead.
    Source code(tar.gz)
    Source code(zip)
  • v1.12.1(Aug 31, 2022)

    • Fix memory leak when using a recursive structure (of tuples or lists) as data in the Model.fit() or the ModelBundle.train_data() methods.
    Source code(tar.gz)
    Source code(zip)
  • v1.12(Jul 16, 2022)

  • v1.11(Apr 28, 2022)

  • v1.10.1(Mar 15, 2022)

    • Major bug (introduced in v1.10) fix: the state of the loss function was not reset after each epoch/evaluate calls so the values returned were averages for the whole lifecycle of the Model class.
    Source code(tar.gz)
    Source code(zip)
  • v1.10(Mar 8, 2022)

    • Add a WandB logger.
    • Epoch and batch metrics are now unified. Their only difference is whether the metric for the batch is computed. The main interface is now the Metric class. It is compatible with TorchMetrics. Thus, TorchMetrics metrics can now be passed as either batch or epoch metrics. The metrics with the interface metric(y_pred, y_true) are internally wrapped into a Metric object and are still fully supported. The torch_metrics keyword argument and the EpochMetric class are now deprecated and will be removed in future versions.
    • Model.get_batch_size is replaced by poutyne.get_batch_size().
    Source code(tar.gz)
    Source code(zip)
  • v1.9(Feb 18, 2022)

    • Add support for TorchMetrics metrics.
    • Experiment is now an alias for ModelBundle, a class quite similar to Experiment except that it allows to instantiate an "Experiment" from a Poutyne Model or a network.
    • Add support for PackedSequence.
    • Add flag to TensorBoardLogger to allow to put training and validation metrics in different graphs. This allow to have a behavior closer to Keras.
    • Add support for fscore on binary classification.
    • Add convert_to_numpy flag to be able to obtain tensors instead of NumPy arrays in evaluate* and predict*.
    Source code(tar.gz)
    Source code(zip)
  • v1.8(Dec 17, 2021)

    Breaking changes:

    • When using epoch metrics 'f1', 'precision', 'recall' and associated classes, the default average has been changed to 'macro' instead of 'micro'. This changes the names of the metrics that is displayed and that is in the log dictionnary in callbacks. This change also applies to Experiment when using task='classif'.
    • Exceptions when loading checkpoints in Experiment are now propagated instead of being silenced.
    Source code(tar.gz)
    Source code(zip)
  • v1.7(Oct 30, 2021)

  • v1.6(Aug 27, 2021)

    • PeriodicSaveCallback and all its subclasses now have the restore_best argument.
    • Experiment now contains a monitoring argument that can be set to false to avoid monitoring any metric and saving uneeded checkpoints.
    • The format of the ETA time and total time now contains days, hours, minutes when appropriate.
    • Add predict methods to Callback to allow callback to be call during prediction phase.
    • Add infer methods to Experiment to more easily make inference (predictions) with an experiment.
    • Add a progress bar callback during predictions of a model.
    • Add a method to compare the results of two experiments.
    • Add return_ground_truth and has_ground_truth arguments to predict_dataset and predict_generator.
    Source code(tar.gz)
    Source code(zip)
  • v1.5(May 22, 2021)

    • Add LambdaCallback to more easily define a callback from lambdas or functions.
    • In Jupyter Notebooks, when coloring is enabled, the print rate of progress output is limited to one output every 0.1 seconds. This solves the slowness problem (and the memory problem on Firefox) when there is a great number of steps per epoch.
    • Add return_dict_format argument to train_on_batch and evaluate_on_batch and allows to return predictions and ground truths in evaluate_* even when return_dict_format=True. Furthermore, Experiment.test* now support return_pred=True and return_ground_truth=True.
    • Split Tips and Tricks example into two examples: Tips and Tricks and Sequence Tagging With an RNN.
    Source code(tar.gz)
    Source code(zip)
  • v1.4(Apr 15, 2021)

    • Add examples for image reconstruction and semantic segmentation with Poutyne.
    • Add the following flags in ProgressionCallback: show_every_n_train_steps, show_every_n_valid_steps, show_every_n_test_steps. They allow to show only certain steps instead of all steps.
    • Fix bug where all warnings were silenced.
    • Add strict flag when loading checkpoints. In Model, a NamedTuple is returned as in PyTorch's load_state_dict. In Experiment, a warning is raised when there are missing or unexpected keys in the checkpoint.
    • In CSVLogger, when multiple learning rates are used, we use the column names lr_group_0, lr_group_1, etc. instead of lr.
    • Fix bug where EarlyStopping would be one epoch late and would anyway disregard the monitored metric at the last epoch.
    Source code(tar.gz)
    Source code(zip)
  • v1.3.1(Mar 5, 2021)

  • v1.3(Feb 11, 2021)

    • A progress bar is now set on validation a model (similar to training). It is disableable by passing progress_options=dict(show_on_valid=False) in the fit* methods.
    • A progress bar is now set testing a model (similar to training). It is disableable by passing verbose=False in the evaluate* methods.
    • A new notification callback NotificationCallback allowing to received message at specific time (start/end training/testing an at any given epoch).
    • A new logging callback, MLflowLogger, this callback allows you to log experimentation configuration and metrics during training, validation and testing.
    • Fix bug where evaluate_generator did not support generators with StopIteration exception.
    • Experiment now has a train_data and a test_data method.
    • The Lambda layer now supports multiple arguments in its forward method.
    Source code(tar.gz)
    Source code(zip)
  • v1.2(Dec 24, 2020)

    • A device argument is added to Model.
    • The argument optimizer of Model can now be a dictionary. This allows to pass different argument to the optimizer, e.g. optimizer=dict(optim='sgd', lr=0.1).
    • The progress bar now uses 20 characters instead of 25.
    • The progress bar is now more fluid since partial blocks are used allowing increments of 1/8th of a block at once.
    • The function torch_to_numpy now does .detach() before .cpu(). This might slightly improves performances in some cases.
    • In Experiment, the load_checkpoint method can now load arbitrary checkpoints by passing a filename instead of the usual argument.
    • Experiment now has a train_dataset and a test_dataset method.
    • Experiment is not considered a beta feature anymore.

    Breaking changes:

    • In evaluate, dataloader_kwargs is now a dictionary keyword argument instead of arbitrary keyword arguments. Other methods are already this way. This was an oversight of the last release.
    Source code(tar.gz)
    Source code(zip)
  • v1.1(Nov 14, 2020)

    • There is now a batch metric TopKAccuracy and it is possible to use them as strings for k in 1 to 10 and 20, 30, …, 100, e.g. 'top5'.
    • Add fit_dataset, evaluate_dataset and predict_dataset methods which allow to pass PyTorch Datasets and creates DataLoader internally. Here is an example with MNIST.
    • Colors now work correctly in Colab.
    • The default colorscheme was changed so that it looks good in Colab, notebooks and command line. The previous one was not readable in Colab.
    • Checkpointing callbacks now don't use the Python tempfile package anymore for the temporary file. The use of this package caused problem when the temp filesystem was not on the same partition as the final destination of the checkpoint. The temporary file is now created at the same place as the final destination. Thus, in most use cases, this will render the use of the temporary_filename argument not necessary. The argument is still available for those who need it.
    • In Experiment, it is not possible to call the method test when training without logging.
    Source code(tar.gz)
    Source code(zip)
  • v1.0.1(Oct 28, 2020)

  • v1.0.0(Sep 18, 2020)

    Version 1.0.0 of Poutyne is here!

    • Output is now very nicely colored and now has a progress bar. Both are disableable with the progress_options arguments. The colorama package needs to be installed to have the colors. See the documentation of the fit method for details.
    • Multi-GPU support: Uses torch.nn.parallel.data_parallel under the hood.
    • Huge update to the documentation with a documentation of metrics and a lot of examples.
    • No need to import framework anymore. Everything now can be imported from poutynedirectly, i.e. from poutyne import whatever_you_want.
    • PeriodicSaveCallbacks (such as ModelCheckpoint) now has a flag keep_only_last_best which allow to only keep the last best checkpoint even when the names differ between epochs.
    • FBeta now supports an ignore_index as in nn.CrossEntropyLoss.
    • Epoch metrics strings 'precision' and 'recall' now available directly without instantiating FBeta.
    • Better ETA estimation in output by weighting more recent batches than older batches.
    • Batch metrics acc and bin_acc now have class counterparts Accuracy and BinaryAccuracy in addition to a reduction keyword argument as in PyTorch.
    • Various bug fixes.
    Source code(tar.gz)
    Source code(zip)
  • v0.8.2(May 31, 2020)

    • Add new callback methods on_test_* to callbacks. Callback can now be passed to the evaluate* methods.
    • New epoch metrics for scikit-learn functions (See documentation of SKLearnMetrics).
    • It is now possible to return multiple metrics for a single batch metric function or epoch metric object. Furthermore, their names can be changed. (See note in documentation of Model class)
    • Computation of batch size is now added for dictionnary inputs and outputs. (See documentation of the new method get_batch_size)
    • Add a lot of type hinting.

    Breaking changes:

    • Ground truths and predictions returned by evaluate_generator and predict_generator are going to be concatenated except when inside custom objects in the next version. A warning is issued in those methods. If the warning is disabled as instructed, the new behavior takes place. (See documentation of evaluate_generator and predict_generator)
    • Names of methods on_batch_begin and on_batch_end changed to on_train_batch_begin and on_train_batch_end respectively. When the old names are used, a warning is issued with backward compatibility added. This backward compatibility will be removed in the next version.
    • EpochMetric classes now have an obligatory reset method.
    • Support of Python 3.5 is dropped. (Anyway, PyTorch was already not supporting it either)
    Source code(tar.gz)
    Source code(zip)
  • v0.7.2(Feb 13, 2020)

    Poutyne is now under LGPLv3 instead of GPLv3.

    Essentially, what this means is that you can now include Poutyne into any proprietary software as long as you are willing to provide the source code and the modifications of Poutyne with your software. The LICENSE file contains more details.

    This is not legal advice. You should consult your lawyer about the implication of the license for your own case.

    Source code(tar.gz)
    Source code(zip)
  • v0.7.1(Feb 12, 2020)

    • Fix a bug introduced in v0.7 when only one of epoch metrics and batch metrics were provided and we would try to concatenate a tuple and a list.
    Source code(tar.gz)
    Source code(zip)
  • v0.7(Feb 10, 2020)

    • Add automatic naming for class object in batch_metrics and epoch_metrics.
    • Add get_saved_epochs method to Experiment
    • optimizer parameter can now be set to None in Modelin the case where there is no need for it.
    • Fixes warning from new PyTorch version.
    • Various improvement of the code.

    Breaking changes:

    • Threshold of the binary_accuracy metric is now 0 instead of 0.5 so that it works using the logits instead of the probabilities.
    • The attribute model of the Model class is now called network instead. A deprecation warning is in place until the next version.
    Source code(tar.gz)
    Source code(zip)
  • v0.6(Sep 30, 2019)

    • Poutyne now has a new logo!
    • Add a beta Experiment class that encapsulates logging and checkpointing callbacks so that it is possible to stop and resume optimization at any time.
    • Add epoch metrics allowing to compute metrics over an epoch that are not decomposable such as F1 scores, precision, recall. While only these former epoch metrics are currently available in Poutyne, epoch metrics can allow to compute the AUROC metric, PCC metric, etc.
    • Support for multiple batches per optimizer step. This allows to have smaller batches that fit in memory instead of a big batch that does not fit while retaining the advantage of the big batch.
    • Add return_ground_truth argument to evaluate_generator.
    • Data loading is now taken into account time for progress estimation.
    • Various doc updates and example finetunings.

    Breaking changes:

    • metrics argument in Model is now deprecated. This argument will be removed in the next version. Use batch_metrics instead.
    • pytoune package is now removed.
    • If steps_per_epoch or validation_steps are greater than the generator length in *_generator methods, then the generator is cycled through instead of stopping as before.
    Source code(tar.gz)
    Source code(zip)
  • v0.5.1(May 2, 2019)

  • v0.5(Mar 24, 2019)

    • Adding a new OptimizerPolicy class allowing to have Phase-based learning rate policies. The two following learning policies are also provided:
      • "Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Rates", Leslie N. Smith, Nicholay Topin, https://arxiv.org/abs/1708.07120
      • "SGDR: Stochastic Gradient Descent with Warm Restarts", Ilya Loshchilov, Frank Hutter, https://arxiv.org/abs/1608.0398
    • Adding of "bin_acc" metric for binary classification in addition to the "accuracy" metric".
    • Adding "time" in callbacks' logs.
    • Various refactoring and small bug fixes.
    Source code(tar.gz)
    Source code(zip)
  • v0.4.1(Aug 8, 2018)

    Breaking changes:

    • Update for PyTorch 0.4.1 (PyTorch 0.4 not supported)
    • Keyword arguments must now be passed with their keyword names in most PyToune functions.

    Non-breaking changes:

    • self.optimizer.zero_grad() is called instead of self.model.zero_grad().
    • Support strings as input for all PyTorch loss functions, metrics and optimizers.
    • Add support for generators that raise the StopIteration exception.
    • Refactor of the Model class (no API break changes).
    • Now using pylint as code style linter.
    • Fix typos in documentation.
    Source code(tar.gz)
    Source code(zip)
  • v0.4(Jun 20, 2018)

    • New usage example using MNIST
    • New *_on_batch methods to Model
    • Every Numpy array is converted into a tensor and vice-versa everywhere it applies i.e. methods return Numpy arrays and can take Numpy arrays as input.
    • New convenient simple layers (Flatten, Identity and Lambda layers)
    • New callbacks to save optimizers and LRSchedulers.
    • New Tensorboard callback.
    • Various bug fixes and improvements.
    Source code(tar.gz)
    Source code(zip)
  • v0.3(May 23, 2018)

    Breaking changes:

    • Update to PyTorch 0.4.0
    • When one or zero metric is used, evaluate and evaluate generator do not return numpy arrays anymore.

    Other changes:

    • Model now offers a to() method to send the PyTorch module and its input to a specified device. (thanks PyTorch 0.4.0)
    • There is now a 'accuracy' metric that can be used as string in the metrics list.
    • Various bug fixes.
    Source code(tar.gz)
    Source code(zip)
  • v0.2.2(May 10, 2018)

    Last release before an upgrade with breaking changes due to the update of PyTorch 0.4.0.

    • Add an on_backward_end callback function
    • Add a ClipNorm callback
    • Fix various bugs.
    Source code(tar.gz)
    Source code(zip)
Owner
GRAAL/GRAIL
Machine Learning Research Group - Université Laval
GRAAL/GRAIL
An implementation of Performer, a linear attention-based transformer, in Pytorch

Performer - Pytorch An implementation of Performer, a linear attention-based transformer variant with a Fast Attention Via positive Orthogonal Random

Phil Wang 900 Dec 22, 2022
270 Dec 24, 2022
PyTorch wrappers for using your model in audacity!

PyTorch wrappers for using your model in audacity!

130 Dec 14, 2022
PyTorch implementations of normalizing flow and its variants.

PyTorch implementations of normalizing flow and its variants.

Tatsuya Yatagawa 55 Dec 01, 2022
Implements pytorch code for the Accelerated SGD algorithm.

AccSGD This is the code associated with Accelerated SGD algorithm used in the paper On the insufficiency of existing momentum schemes for Stochastic O

205 Jan 02, 2023
Tutorial for surrogate gradient learning in spiking neural networks

SpyTorch A tutorial on surrogate gradient learning in spiking neural networks Version: 0.4 This repository contains tutorial files to get you started

Friedemann Zenke 203 Nov 28, 2022
A tutorial on "Bayesian Compression for Deep Learning" published at NIPS (2017).

Code release for "Bayesian Compression for Deep Learning" In "Bayesian Compression for Deep Learning" we adopt a Bayesian view for the compression of

Karen Ullrich 190 Dec 30, 2022
Fast Discounted Cumulative Sums in PyTorch

TODO: update this README! Fast Discounted Cumulative Sums in PyTorch This repository implements an efficient parallel algorithm for the computation of

Daniel Povey 7 Feb 17, 2022
Learning Sparse Neural Networks through L0 regularization

Example implementation of the L0 regularization method described at Learning Sparse Neural Networks through L0 regularization, Christos Louizos, Max W

AMLAB 202 Nov 10, 2022
A PyTorch implementation of L-BFGS.

PyTorch-LBFGS: A PyTorch Implementation of L-BFGS Authors: Hao-Jun Michael Shi (Northwestern University) and Dheevatsa Mudigere (Facebook) What is it?

Hao-Jun Michael Shi 478 Dec 27, 2022
Pytorch implementation of Distributed Proximal Policy Optimization

Pytorch-DPPO Pytorch implementation of Distributed Proximal Policy Optimization: https://arxiv.org/abs/1707.02286 Using PPO with clip loss (from https

Alexis David Jacq 164 Jan 05, 2023
torch-optimizer -- collection of optimizers for Pytorch

torch-optimizer torch-optimizer -- collection of optimizers for PyTorch compatible with optim module. Simple example import torch_optimizer as optim

Nikolay Novik 2.6k Jan 03, 2023
Use Jax functions in Pytorch with DLPack

Use Jax functions in Pytorch with DLPack

Phil Wang 106 Dec 17, 2022
Over9000 optimizer

Optimizers and tests Every result is avg of 20 runs. Dataset LR Schedule Imagenette size 128, 5 epoch Imagewoof size 128, 5 epoch Adam - baseline OneC

Mikhail Grankin 405 Nov 27, 2022
PyTorch implementation of TabNet paper : https://arxiv.org/pdf/1908.07442.pdf

README TabNet : Attentive Interpretable Tabular Learning This is a pyTorch implementation of Tabnet (Arik, S. O., & Pfister, T. (2019). TabNet: Attent

DreamQuark 2k Dec 27, 2022
A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch

Torchmeta A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch. Torchmeta contains popular meta-learning bench

Tristan Deleu 1.7k Jan 06, 2023
A Pytorch Implementation for Compact Bilinear Pooling.

CompactBilinearPooling-Pytorch A Pytorch Implementation for Compact Bilinear Pooling. Adapted from tensorflow_compact_bilinear_pooling Prerequisites I

169 Dec 23, 2022
3D-RETR: End-to-End Single and Multi-View3D Reconstruction with Transformers

3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers (BMVC 2021) Zai Shi*, Zhao Meng*, Yiran Xing, Yunpu Ma, Roger Wattenhofe

Zai Shi 36 Dec 21, 2022
PyTorch framework A simple and complete framework for PyTorch, providing a variety of data loading and simple task solutions that are easy to extend and migrate

PyTorch framework A simple and complete framework for PyTorch, providing a variety of data loading and simple task solutions that are easy to extend and migrate

Cong Cai 12 Dec 19, 2021
On the Variance of the Adaptive Learning Rate and Beyond

RAdam On the Variance of the Adaptive Learning Rate and Beyond We are in an early-release beta. Expect some adventures and rough edges. Table of Conte

Liyuan Liu 2.5k Dec 27, 2022