Elliot is a comprehensive recommendation framework that analyzes the recommendation problem from the researcher's perspective.

Overview

ELLIOT

PyPI - Python Version Version GitHub repo size GitHub

Docs | Paper

Elliot is a comprehensive recommendation framework that analyzes the recommendation problem from the researcher's perspective. It conducts a whole experiment, from dataset loading to results gathering. The core idea is to feed the system with a simple and straightforward configuration file that drives the framework through the experimental setting choices. Elliot untangles the complexity of combining splitting strategies, hyperparameter model optimization, model training, and the generation of reports of the experimental results.

system schema

The framework loads, filters, and splits the data considering a vast set of strategies (splitting methods and filtering approaches, from temporal training-test splitting to nested K-folds Cross-Validation). Elliot optimizes hyperparameters for several recommendation algorithms, selects the best models, compares them with the baselines providing intra-model statistics, computes metrics spanning from accuracy to beyond-accuracy, bias, and fairness, and conducts statistical analysis (Wilcoxon and Paired t-test).

Elliot aims to keep the entire experiment reproducible and put the user in control of the framework.

Installation

Elliot works with the following operating systems:

  • Linux
  • Windows 10
  • macOS X

Elliot requires Python version 3.6 or later.

Elliot requires tensorflow version 2.3.2 or later. If you want to use Elliot with GPU, please ensure that CUDA or cudatoolkit version is 7.6 or later. This requires NVIDIA driver version >= 10.1 (for Linux and Windows10).

Please refer to this document for further working configurations.

Install from source

CONDA

git clone https://github.com//sisinflab/elliot.git && cd elliot
conda create --name elliot_env python=3.8
conda activate elliot_env
pip install --upgrade pip
pip install -e . --verbose

VIRTUALENV

git clone https://github.com//sisinflab/elliot.git && cd elliot
virtualenv -p /usr/bin/python3.6 venv # your python location and version
source venv/bin/activate
pip install --upgrade pip
pip install -e . --verbose

Quick Start

Elliot's entry point is the function run_experiment, which accepts a configuration file that drives the whole experiment. In the following, a sample configuration file is shown to demonstrate how a sample and explicit structure can generate a rigorous experiment.

from elliot.run import run_experiment

run_experiment("configuration/file/path")

The following file is a simple configuration for an experimental setup. It contains all the instructions to get the MovieLens-1M catalog from a specific path and perform a train test split in a random sample way with a ratio of 20%.

This experiment provides a hyperparameter optimization with a grid search strategy for an Item-KNN model. Indeed, it is seen that the possible values of neighbors are closed in squared brackets. It indicates that two different models equipped with two different neighbors' values will be trained and compared to select the best configuration. Moreover, this configuration obliges Elliot to save the recommendation lists with at most 10 items per user as suggest by top_k property.

In this basic experiment, only a simple metric is considered in the final evaluation study. The candidate metric is nDCG for a cutoff equal to top_k, unless otherwise noted.

experiment:
  dataset: movielens_1m
  data_config:
    strategy: dataset
    dataset_path: ../data/movielens_1m/dataset.tsv
  splitting:
    test_splitting:
      strategy: random_subsampling
      test_ratio: 0.2
  models:
    ItemKNN:
      meta:
        hyper_opt_alg: grid
        save_recs: True
      neighbors: [50, 100]
      similarity: cosine
  evaluation:
    simple_metrics: [nDCG]
  top_k: 10

If you want to explore a basic configuration, and an advanced configuration, please refer to:

basic_configuration

advanced_configuration

You can find the full description of the two experiments in the paper.

Contributing

There are many ways to contribute to Elliot! You can contribute code, make improvements to the documentation, report or investigate bugs and issues

We welcome all contributions from bug fixes to new features and extensions.

Feel free to share with us your custom configuration files. We are creating a vault of reproducible experiments, and we would be glad of mentioning your contribution.

Reference Elliot in your blogs, papers, and articles.

Talk about Elliot on social media with the hashtag #elliotrs.

Cite

If you find Elliot useful for your research or development, please cite the following paper:


@article{DBLP:journals/corr/abs-2103-02590,
  author    = {Vito Walter Anelli and
               Alejandro Bellog{\'{\i}}n and
               Antonio Ferrara and
               Daniele Malitesta and
               Felice Antonio Merra and
               Claudio Pomo and
               Francesco M. Donini and
               Tommaso Di Noia},
  title     = {Elliot: a Comprehensive and Rigorous Framework for Reproducible Recommender
               Systems Evaluation},
  journal   = {CoRR},
  volume    = {abs/2103.02590},
  year      = {2021}
}

The Team

Elliot is developed by

It is maintained by SisInfLab Group and Information Retrieval Group.

* Corresponding authors

License

ELLIOT uses APACHE2 License.

Acknowledgements

SliM and an alternative KNN-CF implementation refer to RecSys2019_DeepLearning_Evaluation

Comments
  • Dict Comprehension speeds up data preparation

    Dict Comprehension speeds up data preparation

    Describe the bug I have been using datasets to make experiments with elliot where userId and itemId are strings. In these cases I have noticed speed improvement if I use dict comprehension instead of loops for the preparation of the dataset.

    To Reproduce Link for the commit on local branch. I could create a pull request if you think that it would be an improvement on your side as well.

    import pandas as pd
    import time
    
    data = pd.read_csv('data/cat_dbpedia_movielens_1m/dataset.tsv',sep='\t',header=None)
    data.columns = ['userId','itemId','rating','timestamp']
    #change type of column 
    data.userId = data.userId.astype(str)
    
    ##Loop 1 vs dict comprehension
    t0 = time.time()
    users = list(data['userId'].unique())
    ratings = {}
    for u in users:
        sel_ = data[data['userId'] == u]
        ratings[u] = dict(zip(sel_['itemId'], sel_['rating']))
    t1 = time.time()
    print(t1-t0) #356.07s
    
    t0 = time.time()
    ratings = {k: f.groupby('itemId')['rating'].apply(float).to_dict() for k, f in data.groupby('userId')}
    t1 = time.time()
    print(t1-t0) #29.83
    
    ##Loop 2 vs dict comprehension
    dataframe = data 
    #users are not included in the time loop since they are provided as parameter in the method
    users = list(data['userId'].unique())
    t0 = time.time()
    ratings = {}
    for u in users:
        sel_ = dataframe[dataframe['userId'] == u]
        ratings[u] = dict(zip(sel_['itemId'], sel_['rating']))
    t1 = time.time()
    print(t1-t0) #371.22
    
    t0 = time.time()
    ratings = {k: f.groupby('itemId')['rating'].apply(float).to_dict() for k, f in dataframe.groupby('userId') if k in users}
    t1 = time.time()
    print(t1-t0) #32.44s
    

    Expected behavior I would expect the time required for data preparation to be on the same level regardless of the type of userId/itemId.

    Screenshots

    System details (please complete the following information):

    • OS: Ubuntu 20.04
    • Python: 3.8.10

    Additional context

    enhancement 
    opened by pkaram 3
  • No such file or directory: 'D:\\develop\\elliot\\data\\movielens_1m\\dataset.tsv'

    No such file or directory: 'D:\\develop\\elliot\\data\\movielens_1m\\dataset.tsv'

    Version Number: 0.3.1 2022-10-14 15:42:23.398801: I Start experiment Traceback (most recent call last): 2022-10-14 15:42:23.415288: I There will be the splitting File ".\start_experiments.py", line 8, in run_experiment(f"config_files/{args.config}.yml") File "D:\develop\elliot\elliot\run.py", line 60, in run_experiment dataloader = dataloader_class(config=base.base_namespace) File "D:\develop\elliot\elliot\dataset\dataset.py", line 95, in init self.dataframe = pd.read_csv(path_dataset, sep="\t", header=None, names=self.column_names) File "D:\Users\Anaconda3\envs\elliot_env\lib\site-packages\pandas\io\parsers.py", line 688, in read_csv return _read(filepath_or_buffer, kwds) File "D:\Users\Anaconda3\envs\elliot_env\lib\site-packages\pandas\io\parsers.py", line 454, in _read parser = TextFileReader(fp_or_buf, **kwds) File "D:\Users\Anaconda3\envs\elliot_env\lib\site-packages\pandas\io\parsers.py", line 948, in init self._make_engine(self.engine) File "D:\Users\Anaconda3\envs\elliot_env\lib\site-packages\pandas\io\parsers.py", line 1180, in _make_engine self._engine = CParserWrapper(self.f, **self.options) File "D:\Users\Anaconda3\envs\elliot_env\lib\site-packages\pandas\io\parsers.py", line 2010, in init self._reader = parsers.TextReader(src, **kwds) File "pandas_libs\parsers.pyx", line 382, in pandas._libs.parsers.TextReader.cinit File "pandas_libs\parsers.pyx", line 674, in pandas._libs.parsers.TextReader._setup_parser_source FileNotFoundError: [Errno 2] No such file or directory: 'D:\develop\elliot\data\movielens_1m\dataset.tsv'

    opened by imitn 1
  • Basic Configuration example has different paths for dataset.tsv

    Basic Configuration example has different paths for dataset.tsv

    Describe the bug Running sample basic (https://github.com/sisinflab/elliot/blob/master/sample_basic.py) encounters a problem for file not found. This is because the dataset.tsv is downloaded into data/movilens_1m folder but the configuration file loaded by the same script (https://github.com/sisinflab/elliot/blob/master/config_files/basic_configuration.yml) looks for the dataset in /data/cat_dbpedia_movielens_1m.

    To Reproduce Run sample_basic.py

    Expected behavior dataset.tsv should be downloaded into a folder reachable by the code

    Screenshots

    System details (please complete the following information):

    • OS: [e.g. iOS]
    • Python Version [e.g. 3.6.9]
    • Version of the Libraries [e.g. 22]

    Additional context Add any other context about the problem here.

    opened by karapostK 1
  • Suggestion: SIGIR 2021 E-Commerce Workshop Data Challenge

    Suggestion: SIGIR 2021 E-Commerce Workshop Data Challenge

    Hi! I just discovered this project from links in a discussion area from another project. Having a look through here, I appreciate your features in the area of splitting, unique metrics, and stats tests!

    I learned the other day that the SIGIR 2021 Conference has an e-commerce workshop data challenge, called In-session prediction for purchase intent and recommendations. They just opened it up for participation.

    https://sigir-ecom.github.io/data-task.html

    After looking at elliot's input structure, it seems like it wouldn't be too far off to connect elliot to the data challenge to get some elliot benchmarks out. Though, I didn't dig into the available models enough to know how they map to in-session datasets.

    Unfortunately I am unable to give it a go due to other professional commitments. But I figured I'd share it here as well as an opportunity to grow awareness around elliot, in the event others are interested.

    enhancement 
    opened by tszumowski 1
  • Filenaming problem in Windows 10

    Filenaming problem in Windows 10

    Hello,

    Following the instructions provided in the documentation, I have succesfully installed the Elliot library using Conda v.4.8.2. and Python 3.7.10 on a Windows 10 system.

    However, I have found some problems at the moment of executing some experiments on the framework. The library seems to work finely when I use an algorithm without hyperparameters (as it happens, for instance, with the MostPop algorithm). However, due to how the weight files are named, it does not allow me to execute algorithms with hyperparameters, such as UserKNN or ItemKNN.

    At the moment, the project tries to create a directory or a file (I think is the first one), named algorithm_hyper1:val1_hyper2:val2_..._hyperN:valN where hyperX is the name of the X-th hyperparameter, valX is the value of such hyperparameter and N is the final number of hyperparameters.

    However, Windows does not allow colons in the names of files/folders, so it stops, and returns an OSError as the one I show below: imagen

    Can anything be done to solve this?

    To reproduce I share the configuration file that I used: imagen

    Thanks in advance.

    bug 
    opened by JavierSanzCruza 1
  • FunkSVD finished abruptly

    FunkSVD finished abruptly

    I have tried to run FunkSVD on MovieLens100k but it does finish the execution.

    To Reproduce The screenshots of the running and config files used for reproducing what I did are as follows.

    Screen Shot 2021-03-08 at 13 52 27 Screen Shot 2021-03-08 at 13 52 50

    Expected behavior The screenshot of the execution trace is as follows.

    Screen Shot 2021-03-08 at 13 57 07

    I am Python 3.7 on a MacOS Big Sur version 11.1

    help wanted question 
    opened by lbalby 1
  • Bump tensorflow from 2.3.2 to 2.4.0

    Bump tensorflow from 2.3.2 to 2.4.0

    Bumps tensorflow from 2.3.2 to 2.4.0.

    Release notes

    Sourced from tensorflow's releases.

    TensorFlow 2.4.0

    Release 2.4.0

    Major Features and Improvements

    • tf.distribute introduces experimental support for asynchronous training of models via the tf.distribute.experimental.ParameterServerStrategy API. Please see the tutorial to learn more.

    • MultiWorkerMirroredStrategy is now a stable API and is no longer considered experimental. Some of the major improvements involve handling peer failure and many bug fixes. Please check out the detailed tutorial on Multi-worker training with Keras.

    • Introduces experimental support for a new module named tf.experimental.numpy which is a NumPy-compatible API for writing TF programs. See the detailed guide to learn more. Additional details below.

    • Adds Support for TensorFloat-32 on Ampere based GPUs. TensorFloat-32, or TF32 for short, is a math mode for NVIDIA Ampere based GPUs and is enabled by default.

    • A major refactoring of the internals of the Keras Functional API has been completed, that should improve the reliability, stability, and performance of constructing Functional models.

    • Keras mixed precision API tf.keras.mixed_precision is no longer experimental and allows the use of 16-bit floating point formats during training, improving performance by up to 3x on GPUs and 60% on TPUs. Please see below for additional details.

    • TensorFlow Profiler now supports profiling MultiWorkerMirroredStrategy and tracing multiple workers using the sampling mode API.

    • TFLite Profiler for Android is available. See the detailed guide to learn more.

    • TensorFlow pip packages are now built with CUDA11 and cuDNN 8.0.2.

    Breaking Changes

    • TF Core:

      • Certain float32 ops run in lower precsion on Ampere based GPUs, including matmuls and convolutions, due to the use of TensorFloat-32. Specifically, inputs to such ops are rounded from 23 bits of precision to 10 bits of precision. This is unlikely to cause issues in practice for deep learning models. In some cases, TensorFloat-32 is also used for complex64 ops. TensorFloat-32 can be disabled by running tf.config.experimental.enable_tensor_float_32_execution(False).
      • The byte layout for string tensors across the C-API has been updated to match TF Core/C++; i.e., a contiguous array of tensorflow::tstring/TF_TStrings.
      • C-API functions TF_StringDecode, TF_StringEncode, and TF_StringEncodedSize are no longer relevant and have been removed; see core/platform/ctstring.h for string access/modification in C.
      • tensorflow.python, tensorflow.core and tensorflow.compiler modules are now hidden. These modules are not part of TensorFlow public API.
      • tf.raw_ops.Max and tf.raw_ops.Min no longer accept inputs of type tf.complex64 or tf.complex128, because the behavior of these ops is not well defined for complex types.
      • XLA:CPU and XLA:GPU devices are no longer registered by default. Use TF_XLA_FLAGS=--tf_xla_enable_xla_devices if you really need them, but this flag will eventually be removed in subsequent releases.
    • tf.keras:

      • The steps_per_execution argument in model.compile() is no longer experimental; if you were passing experimental_steps_per_execution, rename it to steps_per_execution in your code. This argument controls the number of batches to run during each tf.function call when calling model.fit(). Running multiple batches inside a single tf.function call can greatly improve performance on TPUs or small models with a large Python overhead.
      • A major refactoring of the internals of the Keras Functional API may affect code that is relying on certain internal details:
        • Code that uses isinstance(x, tf.Tensor) instead of tf.is_tensor when checking Keras symbolic inputs/outputs should switch to using tf.is_tensor.
        • Code that is overly dependent on the exact names attached to symbolic tensors (e.g. assumes there will be ":0" at the end of the inputs, treats names as unique identifiers instead of using tensor.ref(), etc.) may break.
        • Code that uses full path for get_concrete_function to trace Keras symbolic inputs directly should switch to building matching tf.TensorSpecs directly and tracing the TensorSpec objects.
        • Code that relies on the exact number and names of the op layers that TensorFlow operations were converted into may have changed.
        • Code that uses tf.map_fn/tf.cond/tf.while_loop/control flow as op layers and happens to work before TF 2.4. These will explicitly be unsupported now. Converting these ops to Functional API op layers was unreliable before TF 2.4, and prone to erroring incomprehensibly or being silently buggy.
        • Code that directly asserts on a Keras symbolic value in cases where ops like tf.rank used to return a static or symbolic value depending on if the input had a fully static shape or not. Now these ops always return symbolic values.
        • Code already susceptible to leaking tensors outside of graphs becomes slightly more likely to do so now.
        • Code that tries directly getting gradients with respect to symbolic Keras inputs/outputs. Use GradientTape on the actual Tensors passed to the already-constructed model instead.
        • Code that requires very tricky shape manipulation via converted op layers in order to work, where the Keras symbolic shape inference proves insufficient.
        • Code that tries manually walking a tf.keras.Model layer by layer and assumes layers only ever have one positional argument. This assumption doesn't hold true before TF 2.4 either, but is more likely to cause issues now.

    ... (truncated)

    Changelog

    Sourced from tensorflow's changelog.

    Release 2.3.2

    Bug Fixes and Other Changes

    • Fixes an access to unitialized memory in Eigen code (CVE-2020-26266)
    • Fixes a security vulnerability caused by lack of validation in tf.raw_ops.DataFormatVecPermute and tf.raw_ops.DataFormatDimMap (CVE-2020-26267)
    • Fixes a vulnerability caused by attempting to write to immutable memory region in tf.raw_ops.ImmutableConst (CVE-2020-26268
    • Fixes a CHECK-fail in LSTM with zero-length input (CVE-2020-26270)
    • Fixes a security vulnerability caused by accessing heap data outside of bounds when loading a specially crafted SavedModel (CVE-2020-26271)
    • Solves an OOM issue on TPUs when XLA contexts use fused average updates
    • Updates libjpeg-turbo to 2.0.5 to handle CVE-2020-13790.
    • Updates junit to 4.13.1 to handle CVE-2020-15250.
    • Updates PCRE to 8.44 to handle CVE-2019-20838 and CVE-2020-14155.
    • Updates sqlite3 to 3.44.0 to keep in sync with master branch.

    Release 2.2.2

    Bug Fixes and Other Changes

    • Fixes an access to unitialized memory in Eigen code (CVE-2020-26266)
    • Fixes a security vulnerability caused by lack of validation in tf.raw_ops.DataFormatVecPermute and tf.raw_ops.DataFormatDimMap (CVE-2020-26267)
    • Fixes a vulnerability caused by attempting to write to immutable memory region in tf.raw_ops.ImmutableConst (CVE-2020-26268
    • Fixes a CHECK-fail in LSTM with zero-length input (CVE-2020-26270)
    • Fixes a security vulnerability caused by accessing heap data outside of bounds when loading a specially crafted SavedModel (CVE-2020-26271)
    • Prevents memory leaks in loading SavedModels that import functions
    • Updates libjpeg-turbo to 2.0.5 to handle CVE-2020-13790.
    • Updates junit to 4.13.1 to handle CVE-2020-15250.
    • Updates PCRE to 8.44 to handle CVE-2019-20838

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • [Feature Request] Early stopping?

    [Feature Request] Early stopping?

    Not seeing an option for early stopping based on validation metric performance. Running neural models with fixed numbers of epochs is either time-consuming (treating epoch number as a hyperparameter) or risks under/overfitting (fixing the epochs).

    enhancement 
    opened by deklanw 0
  • Fix setup issues

    Fix setup issues

    Fixes https://github.com/sisinflab/elliot/issues/2

    In the latest pip parse_requirements requires a session parameter, and, imports have been shuffled around.

    Also, in the README instructions, activate the newly created conda environment

    Just a note, I imitated the way the code already worked, but relying on the pip internals to do this may not be the best idea https://github.com/pypa/pip/issues/2422

    opened by deklanw 0
  • Setup doesn't work with latest pip

    Setup doesn't work with latest pip

        Running command python setup.py egg_info
        Traceback (most recent call last):
          File "<string>", line 1, in <module>
          File "/home/deklan/code/forks/elliot/setup.py", line 30, in <module>
            install_reqs = parse_requirements('requirements.txt')
        TypeError: parse_requirements() missing 1 required positional argument: 'session'
    

    Pip version

    pip 21.0.1
    
    opened by deklanw 0
  • Evaluration Module: Diversity - Gini Index

    Evaluration Module: Diversity - Gini Index

    Describe the bug According to the algorithm, as it is designed in Elliot, Gini Index can take values > 1, however, according to the bibliography, this is not possible. Could you explain further how is that possible in your algorithm?

    opened by nikosT 0
  • Issues installing on M1 Mac

    Issues installing on M1 Mac

    Please let me know if this has been already discussed before - I've tried finding any information and failed. I have attempted to install Elliot according to the instructions in the docs:

    git clone https://github.com//sisinflab/elliot.git && cd elliot
    virtualenv -p /usr/bin/pyhton3.6 venv # your python location and version
    source venv/bin/activate
    pip install --upgrade pip
    pip install -e . --verbose
    

    I am using a MacBook Pro with M1 Pro chip, 12.0.1 Monterey OS, Python version 3.9 installed with Homebrew. It looks like there is some issue with Cython and some installation quirk, but I really don't know how to approach solving this. Please let me know if anyone else had such an issue previously.

    When I run the last pip install command I get the following terminal output:

    Using pip 22.0.4 from /Users/anastasiia/Desktop/elliot/venv/lib/python3.9/site-packages/pip (python 3.9)
    Looking in indexes: https://pypi.org/simple, https://anastasiia.klimashevskaia%40tv2.no:****@tv2norge.jfrog.io/tv2norge/api/pypi/ai-python-local/simple
    Obtaining file:///Users/anastasiia/Desktop/elliot
      Running command python setup.py egg_info
      running egg_info
      creating /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-pip-egg-info-su39l3jq/elliot.egg-info
      writing /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-pip-egg-info-su39l3jq/elliot.egg-info/PKG-INFO
      writing dependency_links to /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-pip-egg-info-su39l3jq/elliot.egg-info/dependency_links.txt
      writing requirements to /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-pip-egg-info-su39l3jq/elliot.egg-info/requires.txt
      writing top-level names to /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-pip-egg-info-su39l3jq/elliot.egg-info/top_level.txt
      writing manifest file '/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-pip-egg-info-su39l3jq/elliot.egg-info/SOURCES.txt'
      reading manifest file '/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-pip-egg-info-su39l3jq/elliot.egg-info/SOURCES.txt'
      adding license file 'LICENSE'
      writing manifest file '/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-pip-egg-info-su39l3jq/elliot.egg-info/SOURCES.txt'
      Preparing metadata (setup.py) ... done
    Collecting hyperopt==0.2.5
      Using cached hyperopt-0.2.5-py2.py3-none-any.whl (965 kB)
    Collecting numpy==1.18.0
      Using cached numpy-1.18.0.zip (5.4 MB)
      Running command pip subprocess to install build dependencies
      Looking in indexes: https://pypi.org/simple, https://anastasiia.klimashevskaia%40tv2.no:****@tv2norge.jfrog.io/tv2norge/api/pypi/ai-python-local/simple, https://anastasiia.klimashevskaia%40tv2.no:****@tv2norge.jfrog.io/tv2norge/api/pypi/ai-python-local/simple
      Collecting setuptools
        Using cached setuptools-62.1.0-py3-none-any.whl (1.1 MB)
      Collecting wheel
        Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
      Collecting Cython>=0.29.13
        Using cached Cython-0.29.28-py2.py3-none-any.whl (983 kB)
      Installing collected packages: wheel, setuptools, Cython
      Successfully installed Cython-0.29.28 setuptools-62.1.0 wheel-0.37.1
      Installing build dependencies ... done
      Running command Getting requirements to build wheel
      Running from numpy source directory.
      running egg_info
      creating numpy.egg-info
      writing numpy.egg-info/PKG-INFO
      writing dependency_links to numpy.egg-info/dependency_links.txt
      writing entry points to numpy.egg-info/entry_points.txt
      writing top-level names to numpy.egg-info/top_level.txt
      writing manifest file 'numpy.egg-info/SOURCES.txt'
      package init file 'numpy/distutils/mingw/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/include/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/src/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/include/numpy/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/include/numpy/random/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/tests/data/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/src/npysort/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/src/multiarray/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/src/npymath/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/src/common/__init__.py' not found (or not a regular file)
      package init file 'numpy/core/src/umath/__init__.py' not found (or not a regular file)
      package init file 'numpy/linalg/lapack_lite/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/src/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/mixed/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/array_from_pyobj/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/kind/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/common/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/assumed_shape/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/parameter/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/string/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/size/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/tests/src/regression/__init__.py' not found (or not a regular file)
      package init file 'numpy/f2py/src/test/__init__.py' not found (or not a regular file)
      package init file 'numpy/lib/tests/data/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/include/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/_examples/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/src/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/_examples/cffi/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/_examples/cython/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/_examples/numba/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/src/mt19937/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/src/sfc64/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/src/pcg64/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/src/legacy/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/src/philox/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/src/splitmix64/__init__.py' not found (or not a regular file)
      package init file 'numpy/random/src/distributions/__init__.py' not found (or not a regular file)
      package init file 'numpy/_build_utils/src/__init__.py' not found (or not a regular file)
      reading manifest file 'numpy.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      no previously-included directories found matching 'doc/build'
      no previously-included directories found matching 'doc/source/generated'
      no previously-included directories found matching 'benchmarks/env'
      no previously-included directories found matching 'benchmarks/results'
      no previously-included directories found matching 'benchmarks/html'
      no previously-included directories found matching 'benchmarks/numpy'
      warning: no previously-included files matching '*.pyc' found anywhere in distribution
      warning: no previously-included files matching '*.pyo' found anywhere in distribution
      warning: no previously-included files matching '*.pyd' found anywhere in distribution
      warning: no previously-included files matching '*.swp' found anywhere in distribution
      warning: no previously-included files matching '*.bak' found anywhere in distribution
      warning: no previously-included files matching '*~' found anywhere in distribution
      warning: no previously-included files found matching 'LICENSES_bundled.txt'
      adding license file 'LICENSE.txt'
      writing manifest file 'numpy.egg-info/SOURCES.txt'
      Getting requirements to build wheel ... done
      Running command Preparing metadata (pyproject.toml)
      Running from numpy source directory.
      setup.py:425: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
        run_build = parse_setuppy_commands()
      /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/tools/cythonize.py:75: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        required_version = LooseVersion('0.29.13')
      /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/tools/cythonize.py:77: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
        if LooseVersion(cython_version) < required_version:
      Processing numpy/random/_bounded_integers.pxd.in
      Processing numpy/random/_philox.pyx
      Processing numpy/random/_bit_generator.pyx
      Processing numpy/random/_bounded_integers.pyx.in
      Processing numpy/random/_sfc64.pyx
      Processing numpy/random/_mt19937.pyx
      Processing numpy/random/mtrand.pyx
      Processing numpy/random/_generator.pyx
      Processing numpy/random/_pcg64.pyx
      Processing numpy/random/_common.pyx
      Cythonizing sources
      blas_opt_info:
      blas_mkl_info:
      customize UnixCCompiler
        libraries mkl_rt not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      blis_info:
        libraries blis not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      openblas_info:
        libraries openblas not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      atlas_3_10_blas_threads_info:
      Setting PTATLAS=ATLAS
        libraries tatlas not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      atlas_3_10_blas_info:
        libraries satlas not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      atlas_blas_threads_info:
      Setting PTATLAS=ATLAS
        libraries ptf77blas,ptcblas,atlas not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      atlas_blas_info:
        libraries f77blas,cblas,atlas not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      accelerate_info:
        libraries accelerate not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
      Library accelerate was not found. Ignoring
        libraries veclib not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
      Library veclib was not found. Ignoring
        FOUND:
          extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers']
          extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
          define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
    
        FOUND:
          extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers']
          extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
          define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
    
      /bin/sh: svnversion: command not found
      non-existing path in 'numpy/distutils': 'site.cfg'
      lapack_opt_info:
      lapack_mkl_info:
        libraries mkl_rt not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      openblas_lapack_info:
        libraries openblas not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      openblas_clapack_info:
        libraries openblas,lapack not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      flame_info:
        libraries flame not found in ['/Users/anastasiia/Desktop/elliot/venv/lib', '/usr/local/lib', '/usr/lib']
        NOT AVAILABLE
    
      atlas_3_10_threads_info:
      Setting PTATLAS=ATLAS
        libraries lapack_atlas not found in /Users/anastasiia/Desktop/elliot/venv/lib
        libraries tatlas,tatlas not found in /Users/anastasiia/Desktop/elliot/venv/lib
        libraries lapack_atlas not found in /usr/local/lib
        libraries tatlas,tatlas not found in /usr/local/lib
        libraries lapack_atlas not found in /usr/lib
        libraries tatlas,tatlas not found in /usr/lib
      <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
        NOT AVAILABLE
    
      atlas_3_10_info:
        libraries lapack_atlas not found in /Users/anastasiia/Desktop/elliot/venv/lib
        libraries satlas,satlas not found in /Users/anastasiia/Desktop/elliot/venv/lib
        libraries lapack_atlas not found in /usr/local/lib
        libraries satlas,satlas not found in /usr/local/lib
        libraries lapack_atlas not found in /usr/lib
        libraries satlas,satlas not found in /usr/lib
      <class 'numpy.distutils.system_info.atlas_3_10_info'>
        NOT AVAILABLE
    
      atlas_threads_info:
      Setting PTATLAS=ATLAS
        libraries lapack_atlas not found in /Users/anastasiia/Desktop/elliot/venv/lib
        libraries ptf77blas,ptcblas,atlas not found in /Users/anastasiia/Desktop/elliot/venv/lib
        libraries lapack_atlas not found in /usr/local/lib
        libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
        libraries lapack_atlas not found in /usr/lib
        libraries ptf77blas,ptcblas,atlas not found in /usr/lib
      <class 'numpy.distutils.system_info.atlas_threads_info'>
        NOT AVAILABLE
    
      atlas_info:
        libraries lapack_atlas not found in /Users/anastasiia/Desktop/elliot/venv/lib
        libraries f77blas,cblas,atlas not found in /Users/anastasiia/Desktop/elliot/venv/lib
        libraries lapack_atlas not found in /usr/local/lib
        libraries f77blas,cblas,atlas not found in /usr/local/lib
        libraries lapack_atlas not found in /usr/lib
        libraries f77blas,cblas,atlas not found in /usr/lib
      <class 'numpy.distutils.system_info.atlas_info'>
        NOT AVAILABLE
    
        FOUND:
          extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers']
          extra_link_args = ['-Wl,-framework', '-Wl,Accelerate']
          define_macros = [('NO_ATLAS_INFO', 3), ('HAVE_CBLAS', None)]
    
      /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py:275: UserWarning: Unknown distribution option: 'define_macros'
        warnings.warn(msg)
      running dist_info
      running build_src
      build_src
      building py_modules sources
      creating build
      creating build/src.macosx-12-arm64-3.9
      creating build/src.macosx-12-arm64-3.9/numpy
      creating build/src.macosx-12-arm64-3.9/numpy/distutils
      building library "npymath" sources
      Traceback (most recent call last):
        File "/Users/anastasiia/Desktop/elliot/venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
          main()
        File "/Users/anastasiia/Desktop/elliot/venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
        File "/Users/anastasiia/Desktop/elliot/venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 164, in prepare_metadata_for_build_wheel
          return hook(metadata_directory, config_settings)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 188, in prepare_metadata_for_build_wheel
          self.run_setup()
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 281, in run_setup
          super(_BuildMetaLegacyBackend,
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 174, in run_setup
          exec(compile(code, __file__, 'exec'), locals())
        File "setup.py", line 450, in <module>
          setup_package()
        File "setup.py", line 442, in setup_package
          setup(**metadata)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/core.py", line 171, in setup
          return old_setup(**new_attr)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
          return distutils.core.setup(**attrs)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 148, in setup
          return run_commands(dist)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
          dist.run_commands()
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
          self.run_command(cmd)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
          super().run_command(command)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/command/dist_info.py", line 35, in run
          egg_info.run()
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/command/egg_info.py", line 26, in run
          self.run_command("build_src")
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
          self.distribution.run_command(command)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/dist.py", line 1214, in run_command
          super().run_command(command)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
          cmd_obj.run()
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/command/build_src.py", line 146, in run
          self.build_sources()
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/command/build_src.py", line 157, in build_sources
          self.build_library_sources(*libname_info)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/command/build_src.py", line 290, in build_library_sources
          sources = self.generate_sources(sources, (lib_name, build_info))
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/command/build_src.py", line 380, in generate_sources
          source = func(extension, build_dir)
        File "numpy/core/setup.py", line 656, in get_mathlib_info
          st = config_cmd.try_link('int main(void) { return 0;}')
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-build-env-h1wqgh2x/overlay/lib/python3.9/site-packages/setuptools/_distutils/command/config.py", line 241, in try_link
          self._check_compiler()
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/command/config.py", line 82, in _check_compiler
          self.fcompiler = new_fcompiler(compiler=self.fcompiler,
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/__init__.py", line 884, in new_fcompiler
          compiler = get_default_fcompiler(plat, requiref90=requiref90,
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/__init__.py", line 855, in get_default_fcompiler
          compiler_type =  _find_existing_fcompiler(matching_compiler_types,
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/__init__.py", line 806, in _find_existing_fcompiler
          c.customize(dist)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/__init__.py", line 530, in customize
          linker_so_flags = self.flag_vars.linker_so
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/environment.py", line 40, in __getattr__
          return self._get_var(name, conf_desc)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/environment.py", line 56, in _get_var
          var = self._hook_handler(name, hook)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/__init__.py", line 709, in _environment_hook
          return hook()
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/gnu.py", line 353, in get_flags_linker_so
          flags = GnuFCompiler.get_flags_linker_so(self)
        File "/private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc/numpy/distutils/fcompiler/gnu.py", line 144, in get_flags_linker_so
          os.environ['MACOSX_DEPLOYMENT_TARGET'] = target
        File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 684, in __setitem__
          value = self.encodevalue(value)
        File "/opt/homebrew/Cellar/[email protected]/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/os.py", line 756, in encode
          raise TypeError("str expected, not %s" % type(value).__name__)
      TypeError: str expected, not int
      error: subprocess-exited-with-error
    
      × Preparing metadata (pyproject.toml) did not run successfully.
      │ exit code: 1
      ╰─> See above for output.
    
      note: This error originates from a subprocess, and is likely not a problem with pip.
      full command: /Users/anastasiia/Desktop/elliot/venv/bin/python /Users/anastasiia/Desktop/elliot/venv/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py prepare_metadata_for_build_wheel /var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/tmp15w8xm0b
      cwd: /private/var/folders/gr/dpwnfzbj45sfzthq83r6cn7m0000gn/T/pip-install-1_bvhw4u/numpy_94acb307542d4878a85f46abd422e4fc
      Preparing metadata (pyproject.toml) ... error
    error: metadata-generation-failed
    
    × Encountered error while generating package metadata.
    ╰─> See above for output.
    
    note: This is an issue with the package mentioned above, not pip.
    hint: See above for details.
    

    The output of my pip freeze:

    absl-py==1.0.0
    astunparse==1.6.3
    cachetools==5.0.0
    certifi==2021.10.8
    charset-normalizer==2.0.12
    flatbuffers==2.0
    gast==0.5.3
    google-auth==2.6.4
    google-auth-oauthlib==0.4.6
    google-pasta==0.2.0
    grpcio==1.44.0
    h5py==3.6.0
    idna==3.3
    importlib-metadata==4.11.3
    keras==2.8.0
    Keras-Preprocessing==1.1.2
    libclang==13.0.0
    Markdown==3.3.6
    numpy==1.22.3
    oauthlib==3.2.0
    opt-einsum==3.3.0
    protobuf==3.20.0
    pyasn1==0.4.8
    pyasn1-modules==0.2.8
    requests==2.27.1
    requests-oauthlib==1.3.1
    rsa==4.8
    six==1.16.0
    tensorboard==2.8.0
    tensorboard-data-server==0.6.1
    tensorboard-plugin-wit==1.8.1
    tensorflow-macos==2.8.0
    termcolor==1.1.0
    tf-estimator-nightly==2.8.0.dev2021122109
    typing_extensions==4.1.1
    urllib3==1.26.9
    Werkzeug==2.1.1
    wrapt==1.14.0
    zipp==3.8.0
    

    Like I said, I would greatly appreciate any assistance. Thank you!

    opened by anaklim 1
  • Simple metrics are producing different values for the same recommendations

    Simple metrics are producing different values for the same recommendations

    Describe the bug Simple metrics ItemCoverage, UserCoverage, NumRetrieved are producing different values for the same recs/ folder between the procedure of calculating the model and evaluating it and the procedure of loading the model and evaluating it.

    To Reproduce Steps to reproduce the behavior:

    1. Run a model e.g. itemKNN which will produce the recommendations in the results/recs/ dir
    2. Keep evaluation of the running model
    3. Load the results/recs/ dir as a RecommendationFolder
    4. Run evaluation only
    5. Compare the two evaluations

    In both cases, the input dataset has a strategy: fixed. The train.tsv and test.tsv files have previously been produced from a random 0.2 splitting procedure and used as they are for both cases.

    System details (please complete the following information):

    • OS: Debian
    • Python Version 3.8
    • Version of the Libraries installed with conda elliot_env as described in doc
    opened by nikosT 0
  • Sample Scripts Don't Run

    Sample Scripts Don't Run

    I've tried installing the sir_elliot branch as well as the master branch of elliot but none of the existing sample scripts seem to work. I mostly got errors claiming there are no files or directories found. Please advise on how I should proceed or whether there are any alternative frameworks I should use as I need this for a project I'm working on.

    opened by agb2k 0
  • Question

    Question

    Hi,

    Before going more in detail into the implementation of Elliot, I would like to know whether Elliot will be the right package for my experiments.

    Basically, I am interested in context-aware recommender system (CARS) algorithms i.e., FM, FFM. When looking here I found some CARS algorithms but I don't see in the parameter how to add the side information (either user auxilliary information or item auxilliary information). (1) Would that mean that I only need to mention the extra side information in the YAML file only? Or does it mean that there are only specific algorithms implemented in Elliot that do take into account side information? If so could you list them please?

    (2) Do you have somewhere an example of code running CARS algorithm and is it possible to reproduce your results?

    Thank you in advance.

    Looking forward to hearing back from you. Best, -- Manel.

    opened by SlokomManel 0
Releases(v0.3.1)
  • v0.3.1(Jul 8, 2021)

    ELLIOT v0.3.1 Release Notes

    [v0.3.1] - 2021-07-05

    Changed

    • fix AMF: insert of perturbation evaluation, insert multi-step adversarial perturbations
    • fix AMR: insert of perturbation evaluation, insert multi-step adversarial perturbations, improved the data pipeline
    • start experiments with command line parameters (python start_experiments.py --conf name_config_file)
    • released configuration files for DEMO paper (Under review at RecSys 2021)
    • readme file in /data/ directory with links to datasets
    Source code(tar.gz)
    Source code(zip)
  • v0.3.0(Jul 1, 2021)

    ELLIOT v0.3.0 Release Notes

    [v0.3.0] - 2021-06-30

    Changed

    • early stopping strategies
    • offline recommendation files evaluation (ProxyRecommender, RecommendationFolder)
    • negative sampling evaluation
    • improved Microsoft Windows compatibility
    • binarization of explicit dataset
    • automatic loading of implicit datasets
    • multiple prefiltering
    • managing side information with modular loaders
    • alignment of side information with training data
    • improved Documentation: Model creation, Side Information loading, Early Stopping, Negative Sampling
    • added nDCG as formulated in Rendle's 2020 KDD paper
    • visual loader with tensorflow pipeline
    • added and fixing visual recsys method:
      • DVBPR
      • VBPR
      • DeepStyle
      • ACF
      • VNPR
    • added new recommender method
      • MF (Rendle's 2020 RecSys reproducibility paper)
      • EASER
      • RP3beta
      • iALS
    Source code(tar.gz)
    Source code(zip)
  • v0.2.1(Mar 27, 2021)

    ELLIOT v0.2.1 Release Notes

    [v0.2.1] - 2020-03-27

    Changed

    • requirements.txt for Pillow vulnerabilities, change version to >=8.1.1
    • Adversarial features for ECIR tutorial "AL4Rec"
    • Hands-on example for ECIR Tutorial "AL4Rec"
    Source code(tar.gz)
    Source code(zip)
  • v0.2.0(Mar 12, 2021)

    ELLIOT v0.2.0 Release Notes

    Added

    • new dataloader ItemCategoryLoader
    • Enabled FunSVD with batch_size
    • Generalized feature-aware Factorization Machines
    • batch_size field in recommendation snippet examples

    Changed

    • MultiDAE documentation
    • setup issues by @deklanw
    • indentation in README config by @deklanw
    • separated sample main and sample devel scripts
    • paper experiment coherence
    • combination Full recommendation metrics + argpartition
    • Dataset data mock-ups
    • long names namespace model generation during hyperspace exploration
    • default validation k setup
    Source code(tar.gz)
    Source code(zip)
Owner
Information Systems Lab @ Polytechnic University of Bari
Information Systems Lab @ Polytechnic University of Bari
fastFM: A Library for Factorization Machines

Citing fastFM The library fastFM is an academic project. The time and resources spent developing fastFM are therefore justified by the number of citat

1k Dec 24, 2022
Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recommender Systems

DANSER-WWW-19 This repository holds the codes for Dual Graph Attention Networks for Deep Latent Representation of Multifaceted Social Effects in Recom

Qitian Wu 78 Dec 10, 2022
Approximate Nearest Neighbors in C++/Python optimized for memory usage and loading/saving to disk

Annoy Annoy (Approximate Nearest Neighbors Oh Yeah) is a C++ library with Python bindings to search for points in space that are close to a given quer

Spotify 10.6k Jan 01, 2023
QRec: A Python Framework for quick implementation of recommender systems (TensorFlow Based)

QRec is a Python framework for recommender systems (Supported by Python 3.7.4 and Tensorflow 1.14+) in which a number of influential and newly state-of-the-art recommendation models are implemented.

Yu 1.4k Dec 27, 2022
Books Recommendation With Python

Books-Recommendation Business Problem During the last few decades, with the rise

Çağrı Karadeniz 7 Mar 12, 2022
Handling Information Loss of Graph Neural Networks for Session-based Recommendation

LESSR A PyTorch implementation of LESSR (Lossless Edge-order preserving aggregation and Shortcut graph attention for Session-based Recommendation) fro

Tianwen CHEN 62 Dec 03, 2022
Codes for CIKM'21 paper 'Self-Supervised Graph Co-Training for Session-based Recommendation'.

COTREC Codes for CIKM'21 paper 'Self-Supervised Graph Co-Training for Session-based Recommendation'. Requirements: Python 3.7, Pytorch 1.6.0 Best Hype

Xin Xia 43 Jan 04, 2023
Persine is an automated tool to study and reverse-engineer algorithmic recommendation systems.

Persine, the Persona Engine Persine is an automated tool to study and reverse-engineer algorithmic recommendation systems. It has a simple interface a

Jonathan Soma 87 Nov 29, 2022
NVIDIA Merlin is an open source library designed to accelerate recommender systems on NVIDIA’s GPUs.

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in

420 Jan 04, 2023
6002project-rl - An implemention of offline RL on recommender system

An implemention of offline RL on recommender system @author: misajie @update: 20

Tzay Lee 3 May 24, 2022
Beyond Clicks: Modeling Multi-Relational Item Graph for Session-Based Target Behavior Prediction

MGNN-SPred This is our Tensorflow implementation for the paper: WenWang,Wei Zhang, Shukai Liu, Qi Liu, Bo Zhang, Leyu Lin, and Hongyuan Zha. 2020. Bey

Wen Wang 18 Jan 02, 2023
Incorporating User Micro-behaviors and Item Knowledge 59 60 3 into Multi-task Learning for Session-based Recommendation

MKM-SR Incorporating User Micro-behaviors and Item Knowledge into Multi-task Learning for Session-based Recommendation Paper data and code This is the

ciecus 38 Dec 05, 2022
Bundle Graph Convolutional Network

Bundle Graph Convolutional Network This is our Pytorch implementation for the paper: Jianxin Chang, Chen Gao, Xiangnan He, Depeng Jin and Yong Li. Bun

55 Dec 25, 2022
Hierarchical Fashion Graph Network for Personalized Outfit Recommendation, SIGIR 2020

hierarchical_fashion_graph_network This is our Tensorflow implementation for the paper: Xingchen Li, Xiang Wang, Xiangnan He, Long Chen, Jun Xiao, and

LI Xingchen 70 Dec 05, 2022
Spotify API Recommnder System

This project will access your last listened songs on Spotify using its API, then it will request the user to select 5 favorite songs in that list, on which the API will proceed to make 50 recommendat

Kevin Luke 1 Dec 14, 2021
Recommender systems are the systems that are designed to recommend things to the user based on many different factors

Recommender systems are the systems that are designed to recommend things to the user based on many different factors. The recommender system deals with a large volume of information present by filte

Happy N. Monday 3 Feb 15, 2022
RecSim NG: Toward Principled Uncertainty Modeling for Recommender Ecosystems

RecSim NG, a probabilistic platform for multi-agent recommender systems simulation. RecSimNG is a scalable, modular, differentiable simulator implemented in Edward2 and TensorFlow. It offers: a power

Google Research 110 Dec 16, 2022
Mutual Fund Recommender System. Tailor for fund transactions.

Explainable Mutual Fund Recommendation Data Please see 'DATA_DESCRIPTION.md' for mode detail. Recommender System Methods Baseline Collabarative Fiilte

JHJu 2 May 19, 2022
A PyTorch implementation of "Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information" (WSDM 2021)

FairGNN A PyTorch implementation of "Say No to the Discrimination: Learning Fair Graph Neural Networks with Limited Sensitive Attribute Information" (

31 Jan 04, 2023
Attentive Social Recommendation: Towards User And Item Diversities

ASR This is a Tensorflow implementation of the paper: Attentive Social Recommendation: Towards User And Item Diversities Preprint, https://arxiv.org/a

Dongsheng Luo 1 Nov 14, 2021