Python Library for Model Interpretation/Explanations

Overview

Skater

Skater is a unified framework to enable Model Interpretation for all forms of model to help one build an Interpretable machine learning system often needed for real world use-cases(** we are actively working towards to enabling faithful interpretability for all forms models). It is an open source python library designed to demystify the learned structures of a black box model both globally(inference on the basis of a complete data set) and locally(inference about an individual prediction).

The project was started as a research idea to find ways to enable better interpretability(preferably human interpretability) to predictive "black boxes" both for researchers and practioners. The project is still in beta phase.

Install Skater

pip

    Option 1: without rule lists and without deepinterpreter
    pip install -U skater

    Option 2: without rule lists and with deep-interpreter:
    1. Ubuntu: pip3 install --upgrade tensorflow (follow instructions at https://www.tensorflow.org/install/ for details and best practices)
    2. sudo pip install keras
    3. pip install -U skater==1.1.2

    Option 3: For everything included
    1. conda install gxx_linux-64
    2. Ubuntu: pip3 install --upgrade tensorflow (follow instructions https://www.tensorflow.org/install/ for
       details and best practices)
    3. sudo pip install keras
    4. sudo pip install -U --no-deps --force-reinstall --install-option="--rl=True" skater==1.1.2

To get the latest changes try cloning the repo and use the below mentioned commands to get started,


    1. conda install gxx_linux-64
    2. Ubuntu: pip3 install --upgrade tensorflow (follow instructions https://www.tensorflow.org/install/ for
       details and best practices)
    3. sudo pip install keras
    4. git clone the repo
    5. sudo python setup.py install --ostype=linux-ubuntu --rl=True

Testing

  1. If repo is cloned: python skater/tests/all_tests.py
  2. If pip installed: python -c "from skater.tests.all_tests import run_tests; run_tests()"

Usage and Examples

See examples folder for usage examples.

Issues
  • Can't pip install 1.0.3

    Can't pip install 1.0.3

    This version is listed as latest on pypi, but I get this when I try to install it:

    Could not find a version that satisfies the requirement skater==1.0.3 (from versions: 1.0.0b10, 1.0.0b11, 1.0.1, 1.0.2, 1.1.0b1)

    1.0.2 installs just fine (and also installs by default with pip install skater)

    question 
    opened by CPapadim 17
  • Kernel dies when running plot_partial_dependence

    Kernel dies when running plot_partial_dependence

    Hello,

    I'm currently having trouble running plot_partial_dependence for my XGBoost model.

    My kernel dies when I try to run the script, and this happens also if I run partial_dependence: The kernel appears to have died. It will restart automatically.

    In terminal, it says: malloc: *** error for object 0x100007f87ba95b8c: pointer being freed was not allocated *** set a breakpoint in malloc_error_break to debug

    There doesn't seem to be an issue when I run plot_partial_dependence for my random forest model.

    Could someone please look into this? Thanks in advance for your help!

    bug question 
    opened by alay18 16
  • skater.util.exceptions.ModelError: Predict function must be callable

    skater.util.exceptions.ModelError: Predict function must be callable

    Was checking out some examples in the documentation - https://datascienceinc.github.io/Skater/reference/interpretation.html#skater.core.global_interpretation.feature_importance.FeatureImportance.plot_feature_importance

    from skater.model import InMemoryModel
        from skater.core.explanations import Interpretation
        from sklearn.ensemble import RandomForestClassifier
        from sklearn.datasets import load_iris
    
        X, y = load_iris(True)
        rf = RandomForestClassifier()
        rf.fit(X,y)
        model = InMemoryModel(rf, examples = X)
        interpreter = Interpretation()
        interpreter.load_data(X)
        interpreter.feature_importance.plot_feature_importance(model, ascending=True, ax=None)
    

    gave this error,

    Traceback (most recent call last):
      File "rough.py", line 52, in <module>
        main()
      File "rough.py", line 48, in main
        model_interpration_skater()
      File "rough.py", line 40, in model_interpration_skater
        model = InMemoryModel(rf, examples = X)
      File "<ABS_PATH>/lib/python2.7/site-packages/skater/model/local_model.py", line 75, in __init__
        raise(exceptions.ModelError("Predict function must be callable"))
    skater.util.exceptions.ModelError: Predict function must be callable
    
    question 
    opened by Nithanaroy 12
  • Cant install on Python 3.7

    Cant install on Python 3.7

    The installation fails on Python 3.7.3 on a machine running Ubuntu 18.04.3

    Building wheels for collected packages: wordcloud
      Building wheel for wordcloud (setup.py) ... error
      ERROR: Command errored out with exit status 1:
       command: /home/user/.pyenv/versions/3.7.3/envs/absa-py37/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h6dc_hhg/wordcloud/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h6dc_hhg/wordcloud/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-vqaw8gcn --python-tag cp37
           cwd: /tmp/pip-install-h6dc_hhg/wordcloud/
      Complete output (86 lines):
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-3.7
      creating build/lib.linux-x86_64-3.7/wordcloud
      copying wordcloud/__init__.py -> build/lib.linux-x86_64-3.7/wordcloud
      copying wordcloud/color_from_image.py -> build/lib.linux-x86_64-3.7/wordcloud
      copying wordcloud/tokenization.py -> build/lib.linux-x86_64-3.7/wordcloud
      copying wordcloud/wordcloud.py -> build/lib.linux-x86_64-3.7/wordcloud
      copying wordcloud/wordcloud_cli.py -> build/lib.linux-x86_64-3.7/wordcloud
      copying wordcloud/stopwords -> build/lib.linux-x86_64-3.7/wordcloud
      copying wordcloud/DroidSansMono.ttf -> build/lib.linux-x86_64-3.7/wordcloud
      running build_ext
      building 'wordcloud.query_integral_image' extension
      creating build/temp.linux-x86_64-3.7
      creating build/temp.linux-x86_64-3.7/wordcloud
      gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/user/.pyenv/versions/3.7.3/envs/absa-py37/include -I/home/user/.pyenv/versions/3.7.3/include/python3.7m -c wordcloud/query_integral_image.c -o build/temp.linux-x86_64-3.7/wordcloud/query_integral_image.o
      wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSave’:
      wordcloud/query_integral_image.c:14910:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           *type = tstate->exc_type;
                         ^~
      wordcloud/query_integral_image.c:14911:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           *value = tstate->exc_value;
                          ^~
      wordcloud/query_integral_image.c:14912:17: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           *tb = tstate->exc_traceback;
                       ^~
      wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionReset’:
      wordcloud/query_integral_image.c:14924:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           tmp_type = tstate->exc_type;
                            ^~
      wordcloud/query_integral_image.c:14925:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           tmp_value = tstate->exc_value;
                             ^~
      wordcloud/query_integral_image.c:14926:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           tmp_tb = tstate->exc_traceback;
                          ^~
      wordcloud/query_integral_image.c:14927:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           tstate->exc_type = type;
                 ^~
      wordcloud/query_integral_image.c:14928:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           tstate->exc_value = value;
                 ^~
      wordcloud/query_integral_image.c:14929:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           tstate->exc_traceback = tb;
                 ^~
      wordcloud/query_integral_image.c: In function ‘__Pyx_GetException’:
      wordcloud/query_integral_image.c:14972:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           tmp_type = tstate->exc_type;
                            ^~
      wordcloud/query_integral_image.c:14973:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           tmp_value = tstate->exc_value;
                             ^~
      wordcloud/query_integral_image.c:14974:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           tmp_tb = tstate->exc_traceback;
                          ^~
      wordcloud/query_integral_image.c:14975:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           tstate->exc_type = local_type;
                 ^~
      wordcloud/query_integral_image.c:14976:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           tstate->exc_value = local_value;
                 ^~
      wordcloud/query_integral_image.c:14977:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           tstate->exc_traceback = local_tb;
                 ^~
      wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSwap’:
      wordcloud/query_integral_image.c:14999:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           tmp_type = tstate->exc_type;
                            ^~
      wordcloud/query_integral_image.c:15000:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           tmp_value = tstate->exc_value;
                             ^~
      wordcloud/query_integral_image.c:15001:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           tmp_tb = tstate->exc_traceback;
                          ^~
      wordcloud/query_integral_image.c:15002:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
           tstate->exc_type = *type;
                 ^~
      wordcloud/query_integral_image.c:15003:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
           tstate->exc_value = *value;
                 ^~
      wordcloud/query_integral_image.c:15004:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
           tstate->exc_traceback = *tb;
                 ^~
      error: command 'gcc' failed with exit status 1
      ----------------------------------------
      ERROR: Failed building wheel for wordcloud
      Running setup.py clean for wordcloud
    Failed to build wordcloud
    Installing collected packages: wordcloud, Jinja2, soupsieve, beautifulsoup4
      Running setup.py install for wordcloud ... error
        ERROR: Command errored out with exit status 1:
         command: /home/user/.pyenv/versions/3.7.3/envs/absa-py37/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h6dc_hhg/wordcloud/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h6dc_hhg/wordcloud/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ejl6j_1_/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/.pyenv/versions/3.7.3/envs/absa-py37/include/site/python3.7/wordcloud
             cwd: /tmp/pip-install-h6dc_hhg/wordcloud/
        Complete output (86 lines):
        running install
        running build
        running build_py
        creating build
        creating build/lib.linux-x86_64-3.7
        creating build/lib.linux-x86_64-3.7/wordcloud
        copying wordcloud/__init__.py -> build/lib.linux-x86_64-3.7/wordcloud
        copying wordcloud/color_from_image.py -> build/lib.linux-x86_64-3.7/wordcloud
        copying wordcloud/tokenization.py -> build/lib.linux-x86_64-3.7/wordcloud
        copying wordcloud/wordcloud.py -> build/lib.linux-x86_64-3.7/wordcloud
        copying wordcloud/wordcloud_cli.py -> build/lib.linux-x86_64-3.7/wordcloud
        copying wordcloud/stopwords -> build/lib.linux-x86_64-3.7/wordcloud
        copying wordcloud/DroidSansMono.ttf -> build/lib.linux-x86_64-3.7/wordcloud
        running build_ext
        building 'wordcloud.query_integral_image' extension
        creating build/temp.linux-x86_64-3.7
        creating build/temp.linux-x86_64-3.7/wordcloud
        gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -I/home/user/.pyenv/versions/3.7.3/envs/absa-py37/include -I/home/user/.pyenv/versions/3.7.3/include/python3.7m -c wordcloud/query_integral_image.c -o build/temp.linux-x86_64-3.7/wordcloud/query_integral_image.o
        wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSave’:
        wordcloud/query_integral_image.c:14910:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
             *type = tstate->exc_type;
                           ^~
        wordcloud/query_integral_image.c:14911:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
             *value = tstate->exc_value;
                            ^~
        wordcloud/query_integral_image.c:14912:17: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
             *tb = tstate->exc_traceback;
                         ^~
        wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionReset’:
        wordcloud/query_integral_image.c:14924:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
             tmp_type = tstate->exc_type;
                              ^~
        wordcloud/query_integral_image.c:14925:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
             tmp_value = tstate->exc_value;
                               ^~
        wordcloud/query_integral_image.c:14926:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
             tmp_tb = tstate->exc_traceback;
                            ^~
        wordcloud/query_integral_image.c:14927:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
             tstate->exc_type = type;
                   ^~
        wordcloud/query_integral_image.c:14928:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
             tstate->exc_value = value;
                   ^~
        wordcloud/query_integral_image.c:14929:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
             tstate->exc_traceback = tb;
                   ^~
        wordcloud/query_integral_image.c: In function ‘__Pyx_GetException’:
        wordcloud/query_integral_image.c:14972:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
             tmp_type = tstate->exc_type;
                              ^~
        wordcloud/query_integral_image.c:14973:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
             tmp_value = tstate->exc_value;
                               ^~
        wordcloud/query_integral_image.c:14974:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
             tmp_tb = tstate->exc_traceback;
                            ^~
        wordcloud/query_integral_image.c:14975:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
             tstate->exc_type = local_type;
                   ^~
        wordcloud/query_integral_image.c:14976:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
             tstate->exc_value = local_value;
                   ^~
        wordcloud/query_integral_image.c:14977:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
             tstate->exc_traceback = local_tb;
                   ^~
        wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSwap’:
        wordcloud/query_integral_image.c:14999:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
             tmp_type = tstate->exc_type;
                              ^~
        wordcloud/query_integral_image.c:15000:23: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
             tmp_value = tstate->exc_value;
                               ^~
        wordcloud/query_integral_image.c:15001:20: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
             tmp_tb = tstate->exc_traceback;
                            ^~
        wordcloud/query_integral_image.c:15002:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’?
             tstate->exc_type = *type;
                   ^~
        wordcloud/query_integral_image.c:15003:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’?
             tstate->exc_value = *value;
                   ^~
        wordcloud/query_integral_image.c:15004:11: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’?
             tstate->exc_traceback = *tb;
                   ^~
        error: command 'gcc' failed with exit status 1
        ----------------------------------------
    ERROR: Command errored out with exit status 1: /home/user/.pyenv/versions/3.7.3/envs/absa-py37/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-h6dc_hhg/wordcloud/setup.py'"'"'; __file__='"'"'/tmp/pip-install-h6dc_hhg/wordcloud/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ejl6j_1_/install-record.txt --single-version-externally-managed --compile --install-headers /home/user/.pyenv/versions/3.7.3/envs/absa-py37/include/site/python3.7/wordcloud Check the logs for full command output.
    

    It installs just fine on python 3.6.8. Is there any workaround to this?

    opened by zoj613 9
  • Add license to manifest and remove ds-lime requirement for conda-forge

    Add license to manifest and remove ds-lime requirement for conda-forge

    To get the package approved for conda-forge, we need to include the license file in the manifest. Also, there appears to be an orphaned requirement, ds-lime. I've removed it from the setup file. Is this correct?

    Cython is not a requirement either and can also be dropped?

    opened by benvandyke 8
  • Interpretation using Tree Surrogates

    Interpretation using Tree Surrogates

    This idea of explaining model's learned decision policies by extracting rules using a tree based surrogate model is influenced by the work of Mark W. Craven captured as the Trepan algorithm. This PR implements functions to extract the rules as "if else then" hierarchical conditional statements. The results could be visualized or rendered as text both for global and local interpretation.

    Reference papers:

    1. http://ftp.cs.wisc.edu/machine-learning/shavlik-group/craven.thesis.pdf
    2. Extracting Tree structured Representations of Trained Networks: https://papers.nips.cc/paper/1152-extracting-tree-structured-representations-of-trained-networks.pdf
    enhancement v1 new feature 
    opened by pramitchoudhary 7
  • new features

    new features

    • alternative methods for calculating feature importance
    • improvements to datamanager
    • optional training labels
    • feature importance sampling
    opened by aikramer2 6
  • Interpretation using Decision boundaries

    Interpretation using Decision boundaries

    With this PR, the users will be able to visualize the decision boundaries in-regards to an estimator(currently only for Classification)

    This is partial implementation. More work needs to be done to handle multi-dimensional data. Will address that as improvements to this implementation.

    enhancement v1 new feature algorithm 
    opened by pramitchoudhary 6
  • Correction to work with Python 3

    Correction to work with Python 3

    I have a TooManyFeaturesError using plot_partial_dependence with Python 3 that does not occur with Python 2. It seems to be caused by "hasattr(feature_or_feature_pair, 'iter')" in the _check_features function (line 139 of partial_dependence.py). String are iterable with Python 3 but were not with Python 2. Replacing this line by "not isinstance(feature_or_feature_pair, str)" should correct this.

    opened by viticano 6
  • Provide distribution on Conda

    Provide distribution on Conda

    help wanted 
    opened by aikramer2 6
  • Improvement: Improvement to partial_dependence function

    Improvement: Improvement to partial_dependence function

    Too many if else is ugly and error prone and difficult to extend. When time permits lets address this. This will help in reducing bugs. We should do this earlier than later because as we move forward we will forget about things.

    enhancement v2 performance 
    opened by pramitchoudhary 5
  • OverflowError: Range exceeds array bounds

    OverflowError: Range exceeds array bounds

    I run into this problem when I try to use Skater (first time) on a catboost classifier.

    Here is my code:

    from skater.core.explanations import Interpretation
    from skater.model import InMemoryModel
    
    interpreter = Interpretation()
    
    interpreter.load_data(X_train, feature_names=list(X_orig.columns))
    im_model = InMemoryModel(model.predict_proba,
                             examples=X_test, 
                             #target_names=y_train.unique(),
                             #unique_values=y_train.unique()
                            )
    plots = interpreter.feature_importance.plot_feature_importance(im_model, ascending=True)
    

    Here is the output:

    [2/7] features █████--------------- Time elapsed: 0 seconds
    ---------------------------------------------------------------------------
    OverflowError                             Traceback (most recent call last)
    <ipython-input-141-18afbd1218da> in <module>
    ----> 1 plots = interpreter.feature_importance.plot_feature_importance(im_model, ascending=True)
    
    ~\Anaconda3\lib\site-packages\skater\core\global_interpretation\feature_importance.py in plot_feature_importance(self, predict_fn, filter_classes, ascending, ax, progressbar)
        163             raise (MatplotlibDisplayError("Matplotlib unable to open display"))
        164 
    --> 165         importances = self.feature_importance(predict_fn, filter_classes=filter_classes, progressbar=progressbar)
        166 
        167         if ax is None:
    
    ~\Anaconda3\lib\site-packages\skater\core\global_interpretation\feature_importance.py in feature_importance(self, model_instance, ascending, filter_classes, progressbar)
         85 
         86             if self.data_set.feature_info[feature_id]['numeric']:
    ---> 87                 samples = self.data_set.generate_column_sample(feature_id, n_samples=n, method='stratified')
         88             else:
         89                 samples = self.data_set.generate_column_sample(feature_id, n_samples=n, method='random-choice')
    
    ~\Anaconda3\lib\site-packages\skater\data\datamanager.py in generate_column_sample(self, feature_id, n_samples, method)
        407             return self._generate_column_sample_random_choice(feature_id, n_samples=n_samples)
        408         elif method == 'stratified':
    --> 409             return self._generate_column_sample_stratified(feature_id, n_samples=n_samples)
        410         else:
        411             raise(NotImplementedError("Currenly we only support random-choice, stratified for "
    
    ~\Anaconda3\lib\site-packages\skater\data\datamanager.py in _generate_column_sample_stratified(self, feature_id, n_samples, n_bins)
        439         samples = []
        440         for window, n in zip(sample_windows, samples_per_bin):
    --> 441             samples.append(np.random.uniform(window[0], window[1], size=int(n)).tolist())
        442 
        443         return np.array(flatten(samples))
    
    mtrand.pyx in numpy.random.mtrand.RandomState.uniform()
    
    OverflowError: Range exceeds valid bounds
    
    opened by lrq3000 5
  • force exception into using map instead of mapper when n_jobs==1

    force exception into using map instead of mapper when n_jobs==1

    The PartialDependence class currently tries multiprocessing even when n_jobs==1. This code adopted from the FeatureImportance class should fix this from happening.

    opened by alvinthai 5
  • Enabling decision rules with BRL

    Enabling decision rules with BRL

    Skater till now has been an interpretation engine to enable post-hoc model evaluation and interpretation. With this PR Skater starts its journey to support interpretable models. Rule List algorithms are highly popular in the space of Interpretable Models because the trained models are represented as simple decision lists. In this PR, we enable support for Bayesian Rule Lists(BRL). This algorithm helps in building probabilistic classifiers which are composed of pre-mined association rules having a logical sequence of IF - ELSE conditions. The probabilistic classifier( estimating P(Y=1|X) for each X ) optimizes the posterior of a Bayesian hierarchical model over the pre-mined rules.

    References:

    1. Yang et al.(2016) Scalable Bayesian Rule Lists,
      http://proceedings.mlr.press/v70/yang17h/yang17h.pdf
    2. Letham et al.(2015). "Interpretable Classifiers Using Rules and Bayesian Analysis: Building a Better Stroke Prediction Model" https://arxiv.org/pdf/1511.01644.pdf
    3. Freidman et al.(2005) "Predictive Learning via Rule Ensembles" http://statweb.stanford.edu/~jhf/ftp/RuleFit.pdf
    enhancement v1 new feature 
    opened by pramitchoudhary 4
  • Issue 161

    Issue 161

    opened by aikramer2 4
  • Add support for post pruning for Tree Surrogates

    Add support for post pruning for Tree Surrogates

    Decision Trees are non-parametric supervised learning algorithms with several advantages - can handle numerical and categorical features, simpler to interpret(if the tree is of a decent size). However, these learners could get overly complex pretty quickly with non-interpretable trees resulting in overfitting. This could possibly be resolved using pre-pruning and post-pruning. In the previous PR we added support for pre-pruning. With this PR we add support for post-pruning as well. This is a continuation of the effort to enable decision rule based explanation for all forms of models.

    enhancement v1 new feature 
    opened by pramitchoudhary 4
  • Configure private DataScience PyPI server

    Configure private DataScience PyPI server

    This is to probably setup the binaries on the datascience private pypi server for distributing the binaries in a more organized way https://pypi.datascience.com/#/

    opened by pramitchoudhary 4
  • TooManyFeaturesError for partial dependence plot when providing only 1 feature

    TooManyFeaturesError for partial dependence plot when providing only 1 feature

    When attempting to run the example on the readme, a TooManyFeaturesError gets raised even though I am providing only 2 features. The same error gets thrown when I use only 1 feature. The code I ran is below:

    import numpy as np
    from scipy.stats import norm
    
    #gen some data
    B = np.random.normal(0, 10, size = 3)
    X = np.random.normal(0,10, size=(1000, 3))
    feature_names = ["feature_{}".format(i) for i in range(3)]
    e = norm(0, 5)
    y = np.dot(X, B) + e.rvs(1000)
    example = X[0]
    
    #model it
    from sklearn.ensemble import RandomForestRegressor
    regressor = RandomForestRegressor()
    regressor.fit(X, y)
    
    
    #partial dependence
    from skater.core.explanations import Interpretation
    from skater.model import InMemoryModel
    i = Interpretation()
    i.load_data(X, feature_names = feature_names)
    model = InMemoryModel(regressor.predict, examples = X)
    i.partial_dependence.plot_partial_dependence([feature_names[0], feature_names[1]],
                                                model)
    
    ---------------------------------------------------------------------------
    TooManyFeaturesError                      Traceback (most recent call last)
    <ipython-input-29-956960e691ae> in <module>()
         23 model = InMemoryModel(regressor.predict, examples = X)
         24 i.partial_dependence.plot_partial_dependence([feature_names[0], feature_names[1]],
    ---> 25                                             model)
    
    /data/anaconda3/lib/python3.6/site-packages/skater/core/global_interpretation/partial_dependence.py in plot_partial_dependence(self, feature_ids, modelinstance, filter_classes, grid, grid_resolution, grid_range, n_jobs, sample, sampling_strategy, n_samples, bin_count, samples_per_bin, with_variance, figsize)
        506                                                           n_samples=n_samples, bin_count=bin_count,
        507                                                           samples_per_bin=samples_per_bin,
    --> 508                                                           n_jobs=n_jobs, return_metadata=True)
        509 
        510                 self.interpreter.logger.info("done computing pd, now plotting ...")
    
    /data/anaconda3/lib/python3.6/site-packages/skater/core/global_interpretation/partial_dependence.py in partial_dependence(self, feature_ids, modelinstance, filter_classes, grid, grid_resolution, n_jobs, grid_range, sample, sampling_strategy, n_samples, bin_count, samples_per_bin, return_metadata)
        246             raise(exceptions.DataSetNotLoadedError(load_data_not_called_err_msg))
        247 
    --> 248         feature_ids = self._check_features(feature_ids)
        249 
        250         if filter_classes:
    
    /data/anaconda3/lib/python3.6/site-packages/skater/core/global_interpretation/partial_dependence.py in _check_features(self, feature_ids)
        144                                         "use case where you'd like to look at 3 simultaneously" \
        145                                         ", please let us know."
    --> 146             raise(exceptions.TooManyFeaturesError(too_many_features_err_msg))
        147 
        148         if len(feature_ids) == 0:
    
    TooManyFeaturesError: Pass in at most 2 features for pdp. If you have a use case where you'd like to look at 3 simultaneously, please let us know.
    

    However, if I examine the Interpretation object, and call _check_features it behaves as expected:

    i.partial_dependence._check_features(['test1', 'test2'])
    
    out:
    ['test1', 'test2']
    

    My package versions are:

    Python 3.6.0 :: Anaconda custom (64-bit)
    skater                    1.0.0b11
    numpy                     1.13.1
    scikit-learn              0.18.2
    
    bug 
    opened by austinburks 4
  • training labels now bound to data manager (issue 192)

    training labels now bound to data manager (issue 192)

    added this to the api:

    * Datamanager(X, y=y)
    * datamanager.generate_samples(n, include_y=True)
    

    for tasks that require X, y, and sampling.

    opened by aikramer2 4
  • Support classifier.predict

    Support classifier.predict

    enhancement v1 
    opened by aikramer2 4
  • Change api so load data is part of init method

    Change api so load data is part of init method

    just to remove an extra step

    enhancement v1 
    opened by aikramer2 4
  • Use original LIME package together with Skater

    Use original LIME package together with Skater

    Is there any way at all to use the original LIME package with Skater? Basically I need the global interpretation from Skater and the better maintained local interpretation from LIME

    opened by reiinakano 4
  • DataManager fails with Pandas DataFrames

    DataManager fails with Pandas DataFrames

    #241

    Pandas DataFrames have not been properly tests, and calling "feature_info" for a DataManager initialized with a DataFrame raises ValueError.

    This pull-request includes an update to test_data.py, which raises a proper error to show this bug. Am working on a correction.

    opened by bacook17 4
  • platform installation requires system packages

    platform installation requires system packages

    sudo apt-get install libfreetype6-dev libxft-dev related to matplotlib

    opened by aikramer2 4
  • Travis build fails for 3.3/3.4

    Travis build fails for 3.3/3.4

    This is to validate that the library is compatible wrt core functionalities for python 3 as well

    bug 
    opened by pramitchoudhary 4
  • move tests back into package

    move tests back into package

    opened by aikramer2 4
  • Weird halt while running unit_tests on mac

    Weird halt while running unit_tests on mac

    Not sure why at this point in time, but tests on partial_dependence is halting at the following function test_2D_pdp. However it passes on Ubuntu=14.04

    enhancement v2 
    opened by pramitchoudhary 3
  • Simplifying installation

    Simplifying installation

    Moving the dependency on rpy2 from setup.py to setup.sh. This will be helpful in making the installation smoother.

    enhancement v1 new feature Installation 
    opened by pramitchoudhary 3
  • How to install Skater in Kaggle kernel?!

    How to install Skater in Kaggle kernel?!

    I try !pip install skater

    but get:

    `Collecting skater Downloading skater-1.1.2.tar.gz (96 kB) |████████████████████████████████| 96 kB 1.1 MB/s eta 0:00:01 Requirement already satisfied: scikit-learn>=0.18 in /opt/conda/lib/python3.7/site-packages (from skater) (0.23.2) Collecting scikit-image==0.14 Downloading scikit_image-0.14.0-cp37-cp37m-manylinux1_x86_64.whl (25.3 MB) |████████████████████████████████| 25.3 MB 4.4 MB/s eta 0:00:01 Requirement already satisfied: pandas>=0.22.0 in /opt/conda/lib/python3.7/site-packages (from skater) (1.1.2) Collecting ds-lime>=0.1.1.21 Downloading ds-lime-0.1.1.27.tar.gz (253 kB) |████████████████████████████████| 253 kB 59.8 MB/s eta 0:00:01 Requirement already satisfied: requests in /opt/conda/lib/python3.7/site-packages (from skater) (2.23.0) Requirement already satisfied: multiprocess in /opt/conda/lib/python3.7/site-packages (from skater) (0.70.10) Requirement already satisfied: dill>=0.2.6 in /opt/conda/lib/python3.7/site-packages (from skater) (0.3.2) Collecting wordcloud==1.3.1 Downloading wordcloud-1.3.1.tar.gz (169 kB) |████████████████████████████████| 169 kB 62.1 MB/s eta 0:00:01 Collecting joblib==0.11 Downloading joblib-0.11-py2.py3-none-any.whl (176 kB) |████████████████████████████████| 176 kB 60.3 MB/s eta 0:00:01 Collecting Jinja2==2.10 Downloading Jinja2-2.10-py2.py3-none-any.whl (126 kB) |████████████████████████████████| 126 kB 64.0 MB/s eta 0:00:01 Collecting pydotplus==2.0.2 Downloading pydotplus-2.0.2.tar.gz (278 kB) |████████████████████████████████| 278 kB 64.0 MB/s eta 0:00:01 Collecting bs4 Downloading bs4-0.0.1.tar.gz (1.1 kB) Requirement already satisfied: scipy>=0.19.1 in /opt/conda/lib/python3.7/site-packages (from scikit-learn>=0.18->skater) (1.4.1) Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn>=0.18->skater) (2.1.0) Requirement already satisfied: numpy>=1.13.3 in /opt/conda/lib/python3.7/site-packages (from scikit-learn>=0.18->skater) (1.18.5) Requirement already satisfied: pillow>=4.3.0 in /opt/conda/lib/python3.7/site-packages (from scikit-image==0.14->skater) (7.2.0) Requirement already satisfied: six>=1.10.0 in /opt/conda/lib/python3.7/site-packages (from scikit-image==0.14->skater) (1.14.0) Requirement already satisfied: dask[array]>=0.9.0 in /opt/conda/lib/python3.7/site-packages (from scikit-image==0.14->skater) (2.27.0) Requirement already satisfied: cloudpickle>=0.2.1 in /opt/conda/lib/python3.7/site-packages (from scikit-image==0.14->skater) (1.3.0) Requirement already satisfied: PyWavelets>=0.4.0 in /opt/conda/lib/python3.7/site-packages (from scikit-image==0.14->skater) (1.1.1) Requirement already satisfied: matplotlib>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from scikit-image==0.14->skater) (3.2.1) Requirement already satisfied: networkx>=1.8 in /opt/conda/lib/python3.7/site-packages (from scikit-image==0.14->skater) (2.4) Requirement already satisfied: python-dateutil>=2.7.3 in /opt/conda/lib/python3.7/site-packages (from pandas>=0.22.0->skater) (2.8.1) Requirement already satisfied: pytz>=2017.2 in /opt/conda/lib/python3.7/site-packages (from pandas>=0.22.0->skater) (2019.3) Requirement already satisfied: chardet<4,>=3.0.2 in /opt/conda/lib/python3.7/site-packages (from requests->skater) (3.0.4) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /opt/conda/lib/python3.7/site-packages (from requests->skater) (1.24.3) Requirement already satisfied: idna<3,>=2.5 in /opt/conda/lib/python3.7/site-packages (from requests->skater) (2.9) Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.7/site-packages (from requests->skater) (2020.6.20) Requirement already satisfied: MarkupSafe>=0.23 in /opt/conda/lib/python3.7/site-packages (from Jinja2==2.10->skater) (1.1.1) Requirement already satisfied: pyparsing>=2.0.1 in /opt/conda/lib/python3.7/site-packages (from pydotplus==2.0.2->skater) (2.4.7) Requirement already satisfied: beautifulsoup4 in /opt/conda/lib/python3.7/site-packages (from bs4->skater) (4.9.0) Requirement already satisfied: pyyaml in /opt/conda/lib/python3.7/site-packages (from dask[array]>=0.9.0->scikit-image==0.14->skater) (5.3.1) Requirement already satisfied: toolz>=0.8.2; extra == "array" in /opt/conda/lib/python3.7/site-packages (from dask[array]>=0.9.0->scikit-image==0.14->skater) (0.10.0) Requirement already satisfied: cycler>=0.10 in /opt/conda/lib/python3.7/site-packages (from matplotlib>=2.0.0->scikit-image==0.14->skater) (0.10.0) Requirement already satisfied: kiwisolver>=1.0.1 in /opt/conda/lib/python3.7/site-packages (from matplotlib>=2.0.0->scikit-image==0.14->skater) (1.2.0) Requirement already satisfied: decorator>=4.3.0 in /opt/conda/lib/python3.7/site-packages (from networkx>=1.8->scikit-image==0.14->skater) (4.4.2) Requirement already satisfied: soupsieve>1.2 in /opt/conda/lib/python3.7/site-packages (from beautifulsoup4->bs4->skater) (1.9.4) Building wheels for collected packages: skater, ds-lime, wordcloud, pydotplus, bs4 Building wheel for skater (setup.py) ... error ERROR: Command errored out with exit status 1: command: /opt/conda/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5ieiwjh2/skater/setup.py'"'"'; file='"'"'/tmp/pip-install-5ieiwjh2/skater/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-x8i3xny_ cwd: /tmp/pip-install-5ieiwjh2/skater/ Complete output (363 lines): running bdist_wheel running build running build_py creating build creating build/lib creating build/lib/skater copying skater/init.py -> build/lib/skater copying skater/about.py -> build/lib/skater creating build/lib/skater/data copying skater/data/init.py -> build/lib/skater/data copying skater/data/datamanager.py -> build/lib/skater/data creating build/lib/skater/tests copying skater/tests/test_image_ops.py -> build/lib/skater/tests copying skater/tests/init.py -> build/lib/skater/tests copying skater/tests/test_partial_dependence.py -> build/lib/skater/tests copying skater/tests/test_scorer.py -> build/lib/skater/tests copying skater/tests/all_tests.py -> build/lib/skater/tests copying skater/tests/test_data.py -> build/lib/skater/tests copying skater/tests/test_feature_importance.py -> build/lib/skater/tests copying skater/tests/test_dnni.py -> build/lib/skater/tests copying skater/tests/test_validation.py -> build/lib/skater/tests copying skater/tests/test_rule_list.py -> build/lib/skater/tests copying skater/tests/arg_parser.py -> build/lib/skater/tests copying skater/tests/test_tree_surrogates.py -> build/lib/skater/tests copying skater/tests/test_lime.py -> build/lib/skater/tests copying skater/tests/test_model.py -> build/lib/skater/tests copying skater/tests/test_text_ops.py -> build/lib/skater/tests creating build/lib/skater/core copying skater/core/init.py -> build/lib/skater/core copying skater/core/model_interpreter.py -> build/lib/skater/core copying skater/core/validation.py -> build/lib/skater/core copying skater/core/explanations.py -> build/lib/skater/core creating build/lib/skater/model copying skater/model/local_model.py -> build/lib/skater/model copying skater/model/init.py -> build/lib/skater/model copying skater/model/deployed_model.py -> build/lib/skater/model copying skater/model/base.py -> build/lib/skater/model copying skater/model/scorer.py -> build/lib/skater/model creating build/lib/skater/util copying skater/util/image_ops.py -> build/lib/skater/util copying skater/util/text_ops.py -> build/lib/skater/util copying skater/util/init.py -> build/lib/skater/util copying skater/util/kernels.py -> build/lib/skater/util copying skater/util/logger.py -> build/lib/skater/util copying skater/util/dataops.py -> build/lib/skater/util copying skater/util/exceptions.py -> build/lib/skater/util copying skater/util/serialization.py -> build/lib/skater/util copying skater/util/user_defined_types.py -> build/lib/skater/util copying skater/util/plotting.py -> build/lib/skater/util copying skater/util/progressbar.py -> build/lib/skater/util copying skater/util/static_types.py -> build/lib/skater/util creating build/lib/skater/tests/util copying skater/tests/util/build_example.py -> build/lib/skater/tests/util copying skater/tests/util/init.py -> build/lib/skater/tests/util creating build/lib/skater/core/local_interpretation copying skater/core/local_interpretation/init.py -> build/lib/skater/core/local_interpretation copying skater/core/local_interpretation/text_interpreter.py -> build/lib/skater/core/local_interpretation creating build/lib/skater/core/global_interpretation copying skater/core/global_interpretation/init.py -> build/lib/skater/core/global_interpretation copying skater/core/global_interpretation/feature_importance.py -> build/lib/skater/core/global_interpretation copying skater/core/global_interpretation/base.py -> build/lib/skater/core/global_interpretation copying skater/core/global_interpretation/partial_dependence.py -> build/lib/skater/core/global_interpretation copying skater/core/global_interpretation/tree_surrogate.py -> build/lib/skater/core/global_interpretation creating build/lib/skater/core/visualizer copying skater/core/visualizer/text_relevance_visualizer.py -> build/lib/skater/core/visualizer copying skater/core/visualizer/init.py -> build/lib/skater/core/visualizer copying skater/core/visualizer/image_relevance_visualizer.py -> build/lib/skater/core/visualizer copying skater/core/visualizer/tree_visualizer.py -> build/lib/skater/core/visualizer creating build/lib/skater/core/local_interpretation/dnni copying skater/core/local_interpretation/dnni/init.py -> build/lib/skater/core/local_interpretation/dnni copying skater/core/local_interpretation/dnni/initializer.py -> build/lib/skater/core/local_interpretation/dnni copying skater/core/local_interpretation/dnni/deep_interpreter.py -> build/lib/skater/core/local_interpretation/dnni copying skater/core/local_interpretation/dnni/perturbation_relevance_scorer.py -> build/lib/skater/core/local_interpretation/dnni copying skater/core/local_interpretation/dnni/gradient_relevance_scorer.py -> build/lib/skater/core/local_interpretation/dnni creating build/lib/skater/core/local_interpretation/lime copying skater/core/local_interpretation/lime/lime_image.py -> build/lib/skater/core/local_interpretation/lime copying skater/core/local_interpretation/lime/init.py -> build/lib/skater/core/local_interpretation/lime copying skater/core/local_interpretation/lime/lime_tabular.py -> build/lib/skater/core/local_interpretation/lime copying skater/core/local_interpretation/lime/lime_text.py -> build/lib/skater/core/local_interpretation/lime creating build/lib/skater/core/global_interpretation/interpretable_models copying skater/core/global_interpretation/interpretable_models/brlc.py -> build/lib/skater/core/global_interpretation/interpretable_models copying skater/core/global_interpretation/interpretable_models/init.py -> build/lib/skater/core/global_interpretation/interpretable_models copying skater/core/global_interpretation/interpretable_models/bigdatabrlc.py -> build/lib/skater/core/global_interpretation/interpretable_models running egg_info writing skater.egg-info/PKG-INFO writing dependency_links to skater.egg-info/dependency_links.txt writing requirements to skater.egg-info/requires.txt writing top-level names to skater.egg-info/top_level.txt reading manifest file 'skater.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'skater.egg-info/SOURCES.txt' creating build/lib/skater/util/model_specific copying skater/util/model_specific/imagenet_label.json -> build/lib/skater/util/model_specific installing to build/bdist.linux-x86_64/wheel running install Checking .pth file support in build/bdist.linux-x86_64/wheel/ /opt/conda/bin/python3.7 -E -c pass TEST FAILED: build/bdist.linux-x86_64/wheel/ does NOT support .pth files bad install directory or PYTHONPATH

    You are attempting to install a package to a directory that is not on PYTHONPATH and which Python does not read ".pth" files from. The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was:

      build/bdist.linux-x86_64/wheel/
    

    and your PYTHONPATH environment variable currently contains:

      '/kaggle/lib/kagglegym:/kaggle/lib:/kaggle/input/mf-accelerator'
    

    Here are some of your options for correcting the problem:

    • You can choose a different installation directory, i.e., one that is on PYTHONPATH or supports .pth files

    • You can add the installation directory to the PYTHONPATH environment variable. (It must then also be on PYTHONPATH whenever you run Python and want to use the package(s) you are installing.)

    • You can set up the installation directory to support ".pth" files by using one of the approaches described here:

      https://setuptools.readthedocs.io/en/latest/easy_install.html#custom-installation-locations

    Please make the appropriate changes for your system and try again. running bdist_egg installing library code to build/bdist.linux-x86_64/egg running install_lib creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/skater copying build/lib/skater/init.py -> build/bdist.linux-x86_64/egg/skater creating build/bdist.linux-x86_64/egg/skater/data copying build/lib/skater/data/init.py -> build/bdist.linux-x86_64/egg/skater/data copying build/lib/skater/data/datamanager.py -> build/bdist.linux-x86_64/egg/skater/data copying build/lib/skater/about.py -> build/bdist.linux-x86_64/egg/skater creating build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_image_ops.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/init.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_partial_dependence.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_scorer.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/all_tests.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_data.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_feature_importance.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_dnni.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_validation.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_rule_list.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/arg_parser.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_tree_surrogates.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_lime.py -> build/bdist.linux-x86_64/egg/skater/tests copying build/lib/skater/tests/test_model.py -> build/bdist.linux-x86_64/egg/skater/tests creating build/bdist.linux-x86_64/egg/skater/tests/util copying build/lib/skater/tests/util/build_example.py -> build/bdist.linux-x86_64/egg/skater/tests/util copying build/lib/skater/tests/util/init.py -> build/bdist.linux-x86_64/egg/skater/tests/util copying build/lib/skater/tests/test_text_ops.py -> build/bdist.linux-x86_64/egg/skater/tests creating build/bdist.linux-x86_64/egg/skater/core creating build/bdist.linux-x86_64/egg/skater/core/local_interpretation creating build/bdist.linux-x86_64/egg/skater/core/local_interpretation/dnni copying build/lib/skater/core/local_interpretation/dnni/init.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/dnni copying build/lib/skater/core/local_interpretation/dnni/initializer.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/dnni copying build/lib/skater/core/local_interpretation/dnni/deep_interpreter.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/dnni copying build/lib/skater/core/local_interpretation/dnni/perturbation_relevance_scorer.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/dnni copying build/lib/skater/core/local_interpretation/dnni/gradient_relevance_scorer.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/dnni copying build/lib/skater/core/local_interpretation/init.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation creating build/bdist.linux-x86_64/egg/skater/core/local_interpretation/lime copying build/lib/skater/core/local_interpretation/lime/lime_image.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/lime copying build/lib/skater/core/local_interpretation/lime/init.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/lime copying build/lib/skater/core/local_interpretation/lime/lime_tabular.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/lime copying build/lib/skater/core/local_interpretation/lime/lime_text.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation/lime copying build/lib/skater/core/local_interpretation/text_interpreter.py -> build/bdist.linux-x86_64/egg/skater/core/local_interpretation copying build/lib/skater/core/init.py -> build/bdist.linux-x86_64/egg/skater/core copying build/lib/skater/core/model_interpreter.py -> build/bdist.linux-x86_64/egg/skater/core creating build/bdist.linux-x86_64/egg/skater/core/global_interpretation creating build/bdist.linux-x86_64/egg/skater/core/global_interpretation/interpretable_models copying build/lib/skater/core/global_interpretation/interpretable_models/brlc.py -> build/bdist.linux-x86_64/egg/skater/core/global_interpretation/interpretable_models copying build/lib/skater/core/global_interpretation/interpretable_models/init.py -> build/bdist.linux-x86_64/egg/skater/core/global_interpretation/interpretable_models copying build/lib/skater/core/global_interpretation/interpretable_models/bigdatabrlc.py -> build/bdist.linux-x86_64/egg/skater/core/global_interpretation/interpretable_models copying build/lib/skater/core/global_interpretation/init.py -> build/bdist.linux-x86_64/egg/skater/core/global_interpretation copying build/lib/skater/core/global_interpretation/feature_importance.py -> build/bdist.linux-x86_64/egg/skater/core/global_interpretation copying build/lib/skater/core/global_interpretation/base.py -> build/bdist.linux-x86_64/egg/skater/core/global_interpretation copying build/lib/skater/core/global_interpretation/partial_dependence.py -> build/bdist.linux-x86_64/egg/skater/core/global_interpretation copying build/lib/skater/core/global_interpretation/tree_surrogate.py -> build/bdist.linux-x86_64/egg/skater/core/global_interpretation copying build/lib/skater/core/validation.py -> build/bdist.linux-x86_64/egg/skater/core copying build/lib/skater/core/explanations.py -> build/bdist.linux-x86_64/egg/skater/core creating build/bdist.linux-x86_64/egg/skater/core/visualizer copying build/lib/skater/core/visualizer/text_relevance_visualizer.py -> build/bdist.linux-x86_64/egg/skater/core/visualizer copying build/lib/skater/core/visualizer/init.py -> build/bdist.linux-x86_64/egg/skater/core/visualizer copying build/lib/skater/core/visualizer/image_relevance_visualizer.py -> build/bdist.linux-x86_64/egg/skater/core/visualizer copying build/lib/skater/core/visualizer/tree_visualizer.py -> build/bdist.linux-x86_64/egg/skater/core/visualizer creating build/bdist.linux-x86_64/egg/skater/model copying build/lib/skater/model/local_model.py -> build/bdist.linux-x86_64/egg/skater/model copying build/lib/skater/model/init.py -> build/bdist.linux-x86_64/egg/skater/model copying build/lib/skater/model/deployed_model.py -> build/bdist.linux-x86_64/egg/skater/model copying build/lib/skater/model/base.py -> build/bdist.linux-x86_64/egg/skater/model copying build/lib/skater/model/scorer.py -> build/bdist.linux-x86_64/egg/skater/model creating build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/image_ops.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/text_ops.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/init.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/kernels.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/logger.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/dataops.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/exceptions.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/serialization.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/user_defined_types.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/plotting.py -> build/bdist.linux-x86_64/egg/skater/util creating build/bdist.linux-x86_64/egg/skater/util/model_specific copying build/lib/skater/util/model_specific/imagenet_label.json -> build/bdist.linux-x86_64/egg/skater/util/model_specific copying build/lib/skater/util/progressbar.py -> build/bdist.linux-x86_64/egg/skater/util copying build/lib/skater/util/static_types.py -> build/bdist.linux-x86_64/egg/skater/util creating build/bdist.linux-x86_64/egg/EGG-INFO copying skater.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying skater.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying skater.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying skater.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO copying skater.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying skater.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO creating dist creating 'dist/skater-1.1.2-py3.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing skater-1.1.2-py3.7.egg creating /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel/skater-1.1.2-py3.7.egg Extracting skater-1.1.2-py3.7.egg to /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel

    Installed /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel/skater-1.1.2-py3.7.egg Processing dependencies for skater==1.1.2 Searching for bs4 Reading https://pypi.org/simple/bs4/ Downloading https://files.pythonhosted.org/packages/10/ed/7e8b97591f6f456174139ec089c769f89a94a1a4025fe967691de971f314/bs4-0.0.1.tar.gz#sha256=36ecea1fd7cc5c0c6e4a1ff075df26d50da647b75376626cc186e2212886dd3a Best match: bs4 0.0.1 Processing bs4-0.0.1.tar.gz Writing /tmp/easy_install-aa7r89dg/bs4-0.0.1/setup.cfg Running bs4-0.0.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-aa7r89dg/bs4-0.0.1/egg-dist-tmp-vz5t14nm warning: install_lib: 'build/lib' does not exist -- no Python modules to install

    zip_safe flag not set; analyzing archive contents... Moving bs4-0.0.1-py3.7.egg to /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel

    Installed /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel/bs4-0.0.1-py3.7.egg Searching for pydotplus==2.0.2 Reading https://pypi.org/simple/pydotplus/ Downloading https://files.pythonhosted.org/packages/60/bf/62567830b700d9f6930e9ab6831d6ba256f7b0b730acb37278b0ccdffacf/pydotplus-2.0.2.tar.gz#sha256=91e85e9ee9b85d2391ead7d635e3d9c7f5f44fd60a60e59b13e2403fa66505c4 Best match: pydotplus 2.0.2 Processing pydotplus-2.0.2.tar.gz Writing /tmp/easy_install-noh09qix/pydotplus-2.0.2/setup.cfg Running pydotplus-2.0.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-noh09qix/pydotplus-2.0.2/egg-dist-tmp-bl7n4jro zip_safe flag not set; analyzing archive contents... Moving pydotplus-2.0.2-py3.7.egg to /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel

    Installed /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel/pydotplus-2.0.2-py3.7.egg Searching for Jinja2==2.10 Reading https://pypi.org/simple/Jinja2/ Downloading https://files.pythonhosted.org/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl#sha256=74c935a1b8bb9a3947c50a54766a969d4846290e1e788ea44c1392163723c3bd Best match: Jinja2 2.10 Processing Jinja2-2.10-py2.py3-none-any.whl Installing Jinja2-2.10-py2.py3-none-any.whl to /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel

    Installed /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel/Jinja2-2.10-py3.7.egg Searching for joblib==0.11 Reading https://pypi.org/simple/joblib/ Downloading https://files.pythonhosted.org/packages/4f/51/870b2ec270fc29c5d89f85353da420606a9cb39fba4747127e7c7d7eb25d/joblib-0.11-py2.py3-none-any.whl#sha256=cf3420e27048c66916754472bc3a2d4717271103a4806f31f11707a3d82a991f Best match: joblib 0.11 Processing joblib-0.11-py2.py3-none-any.whl Installing joblib-0.11-py2.py3-none-any.whl to /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel

    Installed /tmp/pip-install-5ieiwjh2/skater/build/bdist.linux-x86_64/wheel/joblib-0.11-py3.7.egg Searching for wordcloud==1.3.1 Reading https://pypi.org/simple/wordcloud/ Downloading https://files.pythonhosted.org/packages/be/8b/4e47893d91d2408028b80815a30f128e840104e5ed10999c0121f5f1a7f3/wordcloud-1.3.1.tar.gz#sha256=a5b974e29f7677b66047de2fb214dbed8763c5f5e7b72041ecf2b4040722dce0 Best match: wordcloud 1.3.1 Processing wordcloud-1.3.1.tar.gz Writing /tmp/easy_install-6vv1q_nx/wordcloud-1.3.1/setup.cfg Running wordcloud-1.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-6vv1q_nx/wordcloud-1.3.1/egg-dist-tmp-1xtc6i0e wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSave’: wordcloud/query_integral_image.c:14910:21: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? *type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14911:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? *value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14912:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? *tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionReset’: wordcloud/query_integral_image.c:14924:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14925:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14926:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:14927:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14928:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14929:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = tb; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_GetException’: wordcloud/query_integral_image.c:14972:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14973:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14974:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:14975:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = local_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14976:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = local_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14977:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = local_tb; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSwap’: wordcloud/query_integral_image.c:14999:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:15000:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:15001:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:15002:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = *type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:15003:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = *value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:15004:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = *tb; ^~~~~~~~~~~~~ curexc_traceback error: Setup script exited with error: command 'gcc' failed with exit status 1

    ERROR: Failed building wheel for skater Running setup.py clean for skater Building wheel for ds-lime (setup.py) ... done Created wheel for ds-lime: filename=ds_lime-0.1.1.27-py3-none-any.whl size=261594 sha256=ad7c77aa6f2facd5223349dcf528d6638ac1997bc5d0a97296b60e31f3739ff2 Stored in directory: /root/.cache/pip/wheels/4a/b8/e7/a458f3742328a23a7fc4674207955c47094bc233a9a0139963 Building wheel for wordcloud (setup.py) ... error ERROR: Command errored out with exit status 1: command: /opt/conda/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5ieiwjh2/wordcloud/setup.py'"'"'; file='"'"'/tmp/pip-install-5ieiwjh2/wordcloud/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-iesbirc9 cwd: /tmp/pip-install-5ieiwjh2/wordcloud/ Complete output (107 lines): running bdist_wheel running build running build_py creating build creating build/lib.linux-x86_64-3.7 creating build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/tokenization.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/init.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/color_from_image.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/wordcloud.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/wordcloud_cli.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/stopwords -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/DroidSansMono.ttf -> build/lib.linux-x86_64-3.7/wordcloud running build_ext building 'wordcloud.query_integral_image' extension creating build/temp.linux-x86_64-3.7 creating build/temp.linux-x86_64-3.7/wordcloud gcc -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/conda/include/python3.7m -c wordcloud/query_integral_image.c -o build/temp.linux-x86_64-3.7/wordcloud/query_integral_image.o wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSave’: wordcloud/query_integral_image.c:14910:21: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? *type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14911:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? *value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14912:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? *tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionReset’: wordcloud/query_integral_image.c:14924:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14925:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14926:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:14927:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14928:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14929:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = tb; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_GetException’: wordcloud/query_integral_image.c:14972:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14973:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14974:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:14975:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = local_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14976:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = local_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14977:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = local_tb; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSwap’: wordcloud/query_integral_image.c:14999:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:15000:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:15001:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:15002:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = *type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:15003:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = *value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:15004:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = *tb; ^~~~~~~~~~~~~ curexc_traceback error: command 'gcc' failed with exit status 1

    ERROR: Failed building wheel for wordcloud Running setup.py clean for wordcloud Building wheel for pydotplus (setup.py) ... done Created wheel for pydotplus: filename=pydotplus-2.0.2-py3-none-any.whl size=24566 sha256=266ab5a99d3ea1e725f315dfb523308b2d217658f5095333d1d53d21cc9b5ebb Stored in directory: /root/.cache/pip/wheels/1e/7b/04/7387cf6cc9e48b4a96e361b0be812f0708b394b821bf8c9c50 Building wheel for bs4 (setup.py) ... done Created wheel for bs4: filename=bs4-0.0.1-py3-none-any.whl size=1272 sha256=95f03f807f9a053dcad2789851c36e18fbc2f172105af81f992004565d5b3ee4 Stored in directory: /root/.cache/pip/wheels/0a/9e/ba/20e5bbc1afef3a491f0b3bb74d508f99403aabe76eda2167ca Successfully built ds-lime pydotplus bs4 Failed to build skater wordcloud Installing collected packages: scikit-image, ds-lime, wordcloud, joblib, Jinja2, pydotplus, bs4, skater Attempting uninstall: scikit-image Found existing installation: scikit-image 0.16.2 Uninstalling scikit-image-0.16.2: Successfully uninstalled scikit-image-0.16.2 Attempting uninstall: wordcloud Found existing installation: wordcloud 1.8.0 Uninstalling wordcloud-1.8.0: Successfully uninstalled wordcloud-1.8.0 Running setup.py install for wordcloud ... error ERROR: Command errored out with exit status 1: command: /opt/conda/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5ieiwjh2/wordcloud/setup.py'"'"'; file='"'"'/tmp/pip-install-5ieiwjh2/wordcloud/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-l4_8_zaj/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.7m/wordcloud cwd: /tmp/pip-install-5ieiwjh2/wordcloud/ Complete output (107 lines): running install running build running build_py creating build creating build/lib.linux-x86_64-3.7 creating build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/tokenization.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/init.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/color_from_image.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/wordcloud.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/wordcloud_cli.py -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/stopwords -> build/lib.linux-x86_64-3.7/wordcloud copying wordcloud/DroidSansMono.ttf -> build/lib.linux-x86_64-3.7/wordcloud running build_ext building 'wordcloud.query_integral_image' extension creating build/temp.linux-x86_64-3.7 creating build/temp.linux-x86_64-3.7/wordcloud gcc -pthread -B /opt/conda/compiler_compat -Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -I/opt/conda/include/python3.7m -c wordcloud/query_integral_image.c -o build/temp.linux-x86_64-3.7/wordcloud/query_integral_image.o wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSave’: wordcloud/query_integral_image.c:14910:21: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? *type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14911:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? *value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14912:19: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? *tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionReset’: wordcloud/query_integral_image.c:14924:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14925:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14926:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:14927:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14928:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14929:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = tb; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_GetException’: wordcloud/query_integral_image.c:14972:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14973:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14974:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:14975:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = local_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:14976:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = local_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:14977:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = local_tb; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c: In function ‘__Pyx_ExceptionSwap’: wordcloud/query_integral_image.c:14999:24: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tmp_type = tstate->exc_type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:15000:25: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tmp_value = tstate->exc_value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:15001:22: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tmp_tb = tstate->exc_traceback; ^~~~~~~~~~~~~ curexc_traceback wordcloud/query_integral_image.c:15002:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_type’; did you mean ‘curexc_type’? tstate->exc_type = *type; ^~~~~~~~ curexc_type wordcloud/query_integral_image.c:15003:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_value’; did you mean ‘curexc_value’? tstate->exc_value = *value; ^~~~~~~~~ curexc_value wordcloud/query_integral_image.c:15004:13: error: ‘PyThreadState {aka struct _ts}’ has no member named ‘exc_traceback’; did you mean ‘curexc_traceback’? tstate->exc_traceback = *tb; ^~~~~~~~~~~~~ curexc_traceback error: command 'gcc' failed with exit status 1 ---------------------------------------- Rolling back uninstall of wordcloud Moving to /opt/conda/bin/wordcloud_cli from /tmp/pip-uninstall-by6ks366/wordcloud_cli Moving to /opt/conda/lib/python3.7/site-packages/wordcloud-1.8.0.dist-info/ from /opt/conda/lib/python3.7/site-packages/~ordcloud-1.8.0.dist-info Moving to /opt/conda/lib/python3.7/site-packages/wordcloud/ from /opt/conda/lib/python3.7/site-packages/~ordcloud ERROR: Command errored out with exit status 1: /opt/conda/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5ieiwjh2/wordcloud/setup.py'"'"'; file='"'"'/tmp/pip-install-5ieiwjh2/wordcloud/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record /tmp/pip-record-l4_8_zaj/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.7m/wordcloud Check the logs for full command output. WARNING: You are using pip version 20.2.3; however, version 21.3.1 is available. You should consider upgrading via the '/opt/conda/bin/python3.7 -m pip install --upgrade pip' command.`

    opened by dimka11 1
  • Out of Memory Errors when skater is called

    Out of Memory Errors when skater is called

    As soon as I installed skater for this notebook, the Jupyter NB process starting running out of memory.

    I have 16 GB RAM with a GPU running a recently updated version of Windows 10 on Dell Precision 7530.

    So it should not be having memory issues like this. The last time I tried Oracle's Skater program about 2 years ago I had similar issues.

    Predictive Analytics.ipynb
    Process SpawnPoolWorker-12:
    Process SpawnPoolWorker-1:
    Process SpawnPoolWorker-11:
    Traceback (most recent call last):
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 340, in get
        return _ForkingPickler.loads(res)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 327, in loads
        return load(file, ignore, **kwds)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 313, in load
        return Unpickler(file, ignore=ignore, **kwds).load()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 525, in load
        obj = StockUnpickler.load(self)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 515, in find_class
        return StockUnpickler.find_class(self, module, name)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\__init__.py", line 2, in <module>
        from .core.explanations import Interpretation
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\core\explanations.py", line 3, in <module>
        from .global_interpretation.partial_dependence import PartialDependence
    Traceback (most recent call last):
    Traceback (most recent call last):
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\core\global_interpretation\partial_dependence.py", line 9, in <module>
        from ...data import DataManager
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\data\__init__.py", line 3, in <module>
        from .datamanager import DataManager
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\data\datamanager.py", line 5, in <module>
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\__init__.py", line 82, in <module>
        from .base import clone
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 340, in get
        return _ForkingPickler.loads(res)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 340, in get
        return _ForkingPickler.loads(res)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\base.py", line 17, in <module>
        from .utils import _IS_32BIT
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\utils\__init__.py", line 23, in <module>
        from .class_weight import compute_class_weight, compute_sample_weight
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\utils\class_weight.py", line 7, in <module>
        from .validation import _deprecate_positional_args
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 327, in loads
        return load(file, ignore, **kwds)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 327, in loads
        return load(file, ignore, **kwds)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\utils\validation.py", line 26, in <module>
        from .fixes import _object_dtype_isnan, parse_version
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 313, in load
        return Unpickler(file, ignore=ignore, **kwds).load()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 313, in load
        return Unpickler(file, ignore=ignore, **kwds).load()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\utils\fixes.py", line 20, in <module>
        import scipy.stats
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 525, in load
        obj = StockUnpickler.load(self)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 525, in load
        obj = StockUnpickler.load(self)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\stats\__init__.py", line 388, in <module>
        from .stats import *
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 515, in find_class
        return StockUnpickler.find_class(self, module, name)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 515, in find_class
        return StockUnpickler.find_class(self, module, name)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\stats\stats.py", line 174, in <module>
        from scipy.spatial.distance import cdist
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\spatial\__init__.py", line 101, in <module>
        from ._procrustes import procrustes
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\__init__.py", line 2, in <module>
        from .core.explanations import Interpretation
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\core\explanations.py", line 3, in <module>
        from .global_interpretation.partial_dependence import PartialDependence
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\core\global_interpretation\partial_dependence.py", line 5, in <module>
        import pandas as pd
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\__init__.py", line 30, in <module>
        from pandas._libs import hashtable as _hashtable, lib as _lib, tslib as _tslib
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\_libs\__init__.py", line 3, in <module>
        from .tslibs import (
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\_libs\tslibs\__init__.py", line 3, in <module>
        from .conversion import localize_pydatetime, normalize_date
      File "pandas/_libs/tslibs/c_timestamp.pxd", line 7, in init pandas._libs.tslibs.conversion
      File "pandas/_libs/tslibs/c_timestamp.pyx", line 1, in init pandas._libs.tslibs.c_timestamp
      File "pandas/_libs/tslibs/tzconversion.pyx", line 1, in init pandas._libs.tslibs.tzconversion
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\__init__.py", line 2, in <module>
        from .core.explanations import Interpretation
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\spatial\_procrustes.py", line 9, in <module>
        from scipy.linalg import orthogonal_procrustes
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\linalg\__init__.py", line 211, in <module>
        from ._decomp_update import *
      File "_decomp_update.pyx", line 1, in init scipy.linalg._decomp_update
      File "pandas/_libs/tslibs/timedeltas.pyx", line 1, in init pandas._libs.tslibs.timedeltas
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\core\explanations.py", line 3, in <module>
        from .global_interpretation.partial_dependence import PartialDependence
    ImportError: DLL load failed: The paging file is too small for this operation to complete.
      File "pandas/_libs/tslibs/offsets.pyx", line 1, in init pandas._libs.tslibs.offsets
      File "pandas/_libs/tslibs/ccalendar.pyx", line 12, in init pandas._libs.tslibs.ccalendar
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\_config\__init__.py", line 18, in <module>
        from pandas._config import config
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\core\global_interpretation\partial_dependence.py", line 5, in <module>
        import pandas as pd
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\__init__.py", line 55, in <module>
        from pandas.core.api import (
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\core\api.py", line 24, in <module>
        from pandas.core.groupby import Grouper, NamedAgg
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\core\groupby\__init__.py", line 1, in <module>
        from pandas.core.groupby.generic import (  # noqa: F401
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\_config\config.py", line 54, in <module>
        from typing import Dict, List
      File "<frozen importlib._bootstrap>", line 971, in _find_and_load
      File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 674, in exec_module
      File "<frozen importlib._bootstrap_external>", line 764, in get_code
      File "<frozen importlib._bootstrap_external>", line 833, in get_data
    MemoryError
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\pandas\core\groupby\generic.py", line 44, in <module>
        from pandas.core.frame import DataFrame
      File "<frozen importlib._bootstrap>", line 971, in _find_and_load
      File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 674, in exec_module
      File "<frozen importlib._bootstrap_external>", line 764, in get_code
      File "<frozen importlib._bootstrap_external>", line 833, in get_data
    MemoryError
    Process SpawnPoolWorker-3:
    Process SpawnPoolWorker-5:
    Process SpawnPoolWorker-2:
    Traceback (most recent call last):
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
    Traceback (most recent call last):
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 340, in get
        return _ForkingPickler.loads(res)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 327, in loads
        return load(file, ignore, **kwds)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 313, in load
        return Unpickler(file, ignore=ignore, **kwds).load()
    Traceback (most recent call last):
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 525, in load
        obj = StockUnpickler.load(self)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 340, in get
        return _ForkingPickler.loads(res)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 327, in loads
        return load(file, ignore, **kwds)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 313, in load
        return Unpickler(file, ignore=ignore, **kwds).load()
    MemoryError
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 340, in get
        return _ForkingPickler.loads(res)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 327, in loads
        return load(file, ignore, **kwds)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 313, in load
        return Unpickler(file, ignore=ignore, **kwds).load()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 525, in load
        obj = StockUnpickler.load(self)
    MemoryError
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 525, in load
        obj = StockUnpickler.load(self)
      File "sklearn\tree\_tree.pyx", line 671, in sklearn.tree._tree.Tree.__setstate__
      File "sklearn\tree\_tree.pyx", line 705, in sklearn.tree._tree.Tree._resize_c
      File "sklearn\tree\_utils.pyx", line 41, in sklearn.tree._utils.safe_realloc
    MemoryError: could not allocate 79688 bytes
    INTEL MKL ERROR: The paging file is too small for this operation to complete. mkl_intel_thread.1.dll.
    Intel MKL FATAL ERROR: Cannot load mkl_intel_thread.1.dll.Process SpawnPoolWorker-9:
    
    Traceback (most recent call last):
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 340, in get
        return _ForkingPickler.loads(res)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 327, in loads
        return load(file, ignore, **kwds)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 313, in load
        return Unpickler(file, ignore=ignore, **kwds).load()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 525, in load
        obj = StockUnpickler.load(self)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 515, in find_class
        return StockUnpickler.find_class(self, module, name)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\ensemble\__init__.py", line 8, in <module>
        from ._forest import RandomForestClassifier
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\ensemble\_forest.py", line 56, in <module>
        from ..tree import (DecisionTreeClassifier, DecisionTreeRegressor,
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\tree\__init__.py", line 6, in <module>
        from ._classes import BaseDecisionTree
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\tree\_classes.py", line 41, in <module>
        from ._criterion import Criterion
      File "sklearn\tree\_criterion.pyx", line 1, in init sklearn.tree._criterion
      File "sklearn\tree\_splitter.pyx", line 1, in init sklearn.tree._splitter
      File "sklearn\tree\_tree.pyx", line 1, in init sklearn.tree._tree
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\neighbors\__init__.py", line 17, in <module>
        from ._nca import NeighborhoodComponentsAnalysis
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\neighbors\_nca.py", line 22, in <module>
        from ..decomposition import PCA
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\decomposition\__init__.py", line 8, in <module>
        from ._nmf import NMF, non_negative_factorization
      File "<frozen importlib._bootstrap>", line 971, in _find_and_load
      File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
      File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 674, in exec_module
      File "<frozen importlib._bootstrap_external>", line 771, in get_code
      File "<frozen importlib._bootstrap_external>", line 482, in _validate_bytecode_header
    MemoryError
    [I 2021-08-28 23:54:04.811 ServerApp] Saving file at /Desktop/00-PythonWIP/!0-2021-Sarkar/Ch09/Predictive Analytics.ipynb
    Process SpawnPoolWorker-10:
    Traceback (most recent call last):
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 338, in get
        res = self._reader.recv_bytes()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\connection.py", line 219, in recv_bytes
        buf = self._recv_bytes(maxlength)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\connection.py", line 321, in _recv_bytes
        return self._get_more_data(ov, maxsize)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\connection.py", line 340, in _get_more_data
        assert left > 0
    AssertionError
    Process SpawnPoolWorker-20:
    Traceback (most recent call last):
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 258, in _bootstrap
        self.run()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\process.py", line 93, in run
        self._target(*self._args, **self._kwargs)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\pool.py", line 108, in worker
        task = get()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\multiprocess\queues.py", line 340, in get
        return _ForkingPickler.loads(res)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 327, in loads
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 313, in load
        return Unpickler(file, ignore=ignore, **kwds).load()
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 525, in load
        obj = StockUnpickler.load(self)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\dill\_dill.py", line 515, in find_class
        return StockUnpickler.find_class(self, module, name)
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\__init__.py", line 2, in <module>
        from .core.explanations import Interpretation
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\core\explanations.py", line 3, in <module>
        from .global_interpretation.partial_dependence import PartialDependence
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\core\global_interpretation\partial_dependence.py", line 9, in <module>
        from ...data import DataManager
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\data\__init__.py", line 3, in <module>
        from .datamanager import DataManager
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\skater\data\datamanager.py", line 5, in <module>
        from sklearn.metrics.pairwise import cosine_distances
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\__init__.py", line 82, in <module>
        from .base import clone
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\base.py", line 17, in <module>
        from .utils import _IS_32BIT
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\utils\__init__.py", line 23, in <module>
        from .class_weight import compute_class_weight, compute_sample_weight
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\utils\class_weight.py", line 7, in <module>
        from .validation import _deprecate_positional_args
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\utils\validation.py", line 26, in <module>
        from .fixes import _object_dtype_isnan, parse_version
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\sklearn\utils\fixes.py", line 20, in <module>
        import scipy.stats
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\stats\__init__.py", line 388, in <module>
        from .stats import *
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\stats\stats.py", line 180, in <module>
        from . import distributions
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\stats\distributions.py", line 8, in <module>
        from ._distn_infrastructure import (entropy, rv_discrete, rv_continuous,
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 23, in <module>
        from scipy import optimize
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\optimize\__init__.py", line 387, in <module>
        from .optimize import *
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\optimize\optimize.py", line 36, in <module>
        from ._numdiff import approx_derivative
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\optimize\_numdiff.py", line 6, in <module>
        from scipy.sparse.linalg import LinearOperator
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\sparse\linalg\__init__.py", line 112, in <module>
        from .dsolve import *
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\sparse\linalg\dsolve\__init__.py", line 58, in <module>
        from .linsolve import *
      File "c:\ProgramData\Anaconda3\envs\elyra_env\lib\site-packages\scipy\sparse\linalg\dsolve\linsolve.py", line 11, in <module>
        from . import _superlu
    ImportError: DLL load failed: The paging file is too small for this operation to complete.
    
    
    opened by richlysakowski 0
  • Update PyPI because of outdated requirements

    Update PyPI because of outdated requirements

    The latest released version (PyPI) of Skater is 1.1.2 but since then the changes made to the repo diverge from the PyPI package. Some of these changes include loosening the requirements, which is really helpful when installing with pip and when using it with other packages that have conflicting dependencies. I will be really thankful if you would update the PyPI package.

    opened by Batev 0
  • ValueError: Can't get rows for data of type <class 'pandas.core.series.Series'>

    ValueError: Can't get rows for data of type

    Dear all,

    I am using DecisionTreeClassifier and skater. When I apply skater, I got the error below. ValueError: Can't get rows for data of type <class 'pandas.core.series.Series'> I checked both x_train and x_test are pandas.core.frame.DataFrame while y_train and y_test are numpy.ndarray. I do not know where is the problem.. Any help please? Thanks in advance.

    opened by amaa11 1
  • ModuleNotFoundError: No module named 'skater.core.global_interpretation.tree_surrogate'

    ModuleNotFoundError: No module named 'skater.core.global_interpretation.tree_surrogate'

    I'm using Skater on windows 64 with python 3.6 in Anaconda. I'm trying to implement the example on sklearn's breast cancer dataset (https://github.com/oracle/Skater/blob/master/examples/explaining_classification_breast_cancer.ipynb). When I run the command 'from skater.core.global_interpretation.tree_surrogate import TreeSurrogate' I get the error ModuleNotFoundError: No module named 'skater.core.global_interpretation.tree_surrogate'. Does anyone know why? modulenotfounderror

    opened by aliceschi93 1
  • TypeError: 'NoneType' object is not subscriptable - A problem with the n_samples parameter in plot_feature_importance

    TypeError: 'NoneType' object is not subscriptable - A problem with the n_samples parameter in plot_feature_importance

    I have come across an error in the plot_feature_importance method, relating to the n_samples parameter. When the parameter is set lower than the size of the test set (or train set if the Interpretation object is based on that), the TypeError: 'NoneType' object is not subscriptable, occurs. As long as the n_samples parameter is set higher than the size of the relevant dataset, everything works fine.

    The code in the attached pdf was run in a Google Colab environment, with python 3.6 and skater 1.1.2. I followed the example from your documentation: https://github.com/oracle/Skater/blob/master/examples/explaining_classification_breast_cancer.ipynb

    When applying a much larger dataset than the one in the example, the problem is more critical, as plotting feature importance with a large sample size takes a very long time.

    feature_importance_error.pdf

    opened by OivindBirk 0
  • Cannot install skater with BRLC from MacOS

    Cannot install skater with BRLC from MacOS

    I am unable to get the Bayesian Rule List classifier to work from MacOS so far. I tried:

    • Installing skater with conda -> latest version does not include BRLC
    • Installing with pip using sudo pip install -U --no-deps --force-reinstall --install-option="--rl=True" skater==1.1.2 -> I get:
          File "/opt/anaconda3/envs/skater/lib/python3.6/subprocess.py", line 438, in run
            output=stdout, stderr=stderr)
        subprocess.CalledProcessError: Command '['bash', '/private/var/folders/sb/l953ckxx4v3ds19bcb9n6q400000gr/T/pip-install-qt4_g4a7/skater/setup.sh', 'linux-ubuntu']' returned non-zero exit status 1.
    

    It seems that the pip-install refers to ubuntu, so perhaps MacOS is not supported?

    I also installed skater with pip without any extra options (ensuring Python version <= 3.6, because higher is not supported). That install did work, but when using BRLC, the sbrl R package could not be found. Trying to install it from R resulted in:

    Warning in install.packages :
      package ‘sbrl’ is not available (for R version 3.6.3)
    

    Any suggestions?

    opened by snthibaud 4
  • Feature importance does not sum up to one

    Feature importance does not sum up to one

    I'm getting this error when I train on the classifier models on my own dataset. I also set the number of estimators under control, so that I'm sure it sums up to one. But I get same error. Any help would be appreciated.

    opened by mh2t 0
  • Stuck on the progress bar warning message when using global Interpretation on keras sequential model

    Stuck on the progress bar warning message when using global Interpretation on keras sequential model

    I was able to use skater library functions without much problems for other flavours and models like Random forest, Gradient Booster, Decision Tree etc. to create PDPs and Feature Importances

    However, when implementing the same for Sequential Models using keras library, kernel keeps on running with just the initial warning message.

    from skater.core.explanations import Interpretation
    from skater.model import InMemoryModel
    interpreter = Interpretation(X[10:],Y[10:],list(train_df.columns)[:-1])
    pyintmodel = InMemoryModel(my_model.predict, examples=X[10:], feature_names = list(train_df.columns)[:-1])
    fi = interpreter.partial_dependence.plot_partial_dependence([0,1], pyintmodel)
    
    2020-03-12 16:21:20,483 - skater.core.explanations - WARNING - Progress bars slow down runs by 10-20%. For slightly 
    faster runs, do progressbar=False
    
    

    Is the keras library / neural network models not supported in skater ? Also I am attaching the csv files and my jupyter notebook if that helps ( change extension from txt to ipynb and csv respectively) hourly_wages_data.txt

    Wages_NN_Skater.txt

    opened by deveshcode 0
  • ModuleNotFoundError: No module named 'skater'

    ModuleNotFoundError: No module named 'skater'

    Dear all,

    I try to install skater using pip3.6 install -U skater. I have Python(3.6.9).

    I get the following build error (the error is found in the attached link) after running the aforementioned command [ https://www.codepile.net/pile/xO9A25O6 ].

    Afterward, when I try to import skater I got an error that there is no module called skater. Any ideas how to overcome this issue.

    Thanks.

    opened by Dola47 1
Releases(v1.1.2)
  • v1.1.2(Sep 21, 2018)

    • Support for Tree Surrogates for explanations(Experimental)
    • Bug-fixes
    • Updated notebooks examples
    • Basic documentation update to reflect new changes(more work needs to be done there)
    • Other bug-fixes and improvements
    Source code(tar.gz)
    Source code(zip)
  • v1.1.1-b3(Jul 24, 2018)

    • Simplified Installation to handle dependencies better
    • Convenience functions to support flipping image orientation flip_orientation
    • Updated examples on how to use skater with xgboost
    • Enabled support for Image Inference using Occlusion with the ability to specify window_size and step_size for perturbing feature space
    • Other minor bug fixes and code clean-ups

    Credits: To all contributors who have helped in moving the project forward.

    Source code(tar.gz)
    Source code(zip)
  • v1.1.1-b2(May 17, 2018)

    New Features:

    • Added new interface skater.core.local_interpretation.dnni.deep_interpreter.DeepInterpreter for interpreting tensorflow and Keras based models
    • enabling support for interpreting DNNs using gradient-based e-LRP and Integrated Gradient through DeepInterpreter.explain
    • added support to visualize relevance/attribution scores for interpreting image and text inputs
      • skater.core.visualizer.image_relevance_visualizer.visualize
      • skater.core.visualizer.text_relevance_visualizer import build_visual_explainer, show_in_notebook
    • user-friendly Utility functions to generate simple yet effective conditional adversarial examples for image
inputs
      • skater.util.image_ops import load_image, show_image, normalize, add_noise, flip_pixels, image_transformation
      • skater.util.image_ops import in_between, greater_than, greater_than_or_equal
    • More interactive notebook use-cases for building and interpreting DNNs for evaluating model stability/identifying blind spots
    • Updates to documentation, https://datascienceinc.github.io/Skater/overview.html
    • New section summarizing Notebook examples https://datascienceinc.github.io/Skater/gallery.html
    • Other bug fixes

    Credits:

    • Special thanks to Marco Ancona(@marcoancona) for guiding in enabling this feature within Skater.
    • Thanks to all other contributors for helping move the library forward every day.
    Source code(tar.gz)
    Source code(zip)
  • v1.1.0(May 12, 2018)

    • This is a follow-up release on experimental support for building rule-based models to enable interpretability both at the Global and Local scope.
    • More improved docs with a section on Jupyter Notebook highlighting different supported algorithms.

    More improvement in planned in the subsequent release. Stay tuned.

    Source code(tar.gz)
    Source code(zip)
  • v1.1.0-b1(Mar 15, 2018)

    1. Skater till now has been an interpretation engine to enable post-hoc model evaluation and interpretation. With this PR Skater starts its journey to support interpretable models. Rule List algorithms are highly popular in the space of Interpretable Models because the trained models are represented as simple decision lists. In the latest release, we enable support for Bayesian Rule Lists(BRL). The probabilistic classifier( estimating P(Y=1|X) for each X ) optimizes the posterior of a Bayesian hierarchical model over the pre-mined rules.

      Usage Example:

      from skater.core.global_interpretation.interpretable_models.brlc import BRLC
      import pandas as pd
      from sklearn.datasets.mldata import fetch_mldata
      input_df = fetch_mldata("diabetes")
      ...
      Xtrain, Xtest, ytrain, ytest = train_test_split(input_df, y, test_size=0.20, random_state=0)
      
      sbrl_model = BRLC(min_rule_len=1, max_rule_len=10, iterations=10000, n_chains=20, drop_features=True)
      # Train a model, by default discretizer is enabled. So, you wish to exclude features then exclude them using
      # the undiscretize_feature_list parameter
      model = sbrl_model.fit(Xtrain, ytrain, bin_labels="default")
      
    2. Other minor bug fixes and documentation update

    Credits: Special thanks to Professor Cynthia Rudin, Hongyu Yang and @tmadl(Tamas Madl) for helping enable this feature.

    Source code(tar.gz)
    Source code(zip)
  • v1.0.3(Oct 12, 2017)

    This release includes:

    • Various bug fixes and performance improvements
    • new feature importance calculation method
    • introduction of model scorers
    • model types can now be determined explicitly

    Model Scoring

    Now, after you create a Skater model with:

    model = InMemoryModel(predict_fn, examples=examples, model_type="classifier")
    

    The model object now provides a .scorers api, which allows you to store predictions against training labels. Based on whether your model is a regressor, classifier that returns labels, or classifier that returns probabilities, scorers will automatically expose various scoring algorithms specific to your model. For instance, in the example above, we could do:

    model.scorers.f1(labels, model(X))
    model.scorers.cross_entropy(labels, model(X))
    

    if it were a regression, we could do:

    model.scorers.mse(labels, model(X))
    

    Calling model.scorers.default(labels, model(X)) or simply model.scorers(labels, model(X)) will execute the default scorer for your model, which are:

    regression: mean absolute error classifier (probabilities): cross entropy classifier (labels): f1

    Let us know if you'd like more scorers, or even better, feel free to make a PR to add more yourself!

    Feature Importance Calculation

    The default method of computing feature importance is done by perturbing each feature, and observing how much those perturbations affect predictions.

    With the addition of model scoring, we now also provide a method based on observing changes in model scoring functions; the less accurate your model becomes based on perturbing a feature, the more important it is.

    To enable scoring based feature importance, you must load training labels into your interpretation object, like:

    interpreter = Interpretation(training_data=training_data, training_labels=training_labels)
    interpreter.feature_importance.plot_feature_importance(model, method='model-scoring')
    

    Explicit Model Types

    Originall Skater tried to infer the type of your model based on the types of predictions it made. Now when you create a model, you can define these explicitely with model_type and probability keyword arguments to skater model types:

    model = InMemoryModel(predict_fn, model_type='classifier', probability=True)
    

    or

    model = InMemoryModel(predict_fn, model_type='regressor')
    
    Source code(tar.gz)
    Source code(zip)
  • v1.0.2(Jul 5, 2017)

  • v1.0.0b.12(Jun 15, 2017)

  • v1.0.0b.11(Jun 8, 2017)

  • v1.0.0b.10(May 19, 2017)

    • Beta release of the Model Interpretation library. This is the first public release of the library. So, some of the apis and functionalities could still be experimental
      • Global Interpretation - PDP, Variable Importance
      • Local Interpretation - improved LIME
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0b.2(May 14, 2017)

  • v1.0.0b.1(May 12, 2017)

  • v1.0.0b(May 9, 2017)

    • Support for classifiers without probability scores
    • Support for categorical variables
    • minor code restructuring
    • Column wise stratified sampling support
    • Ability to filter results to subset of classes
    Source code(tar.gz)
    Source code(zip)
  • v1.0.0-alpha(Mar 30, 2017)

Code for "High-Precision Model-Agnostic Explanations" paper

Anchor This repository has code for the paper High-Precision Model-Agnostic Explanations. An anchor explanation is a rule that sufficiently “anchors”

Marco Tulio Correia Ribeiro 685 Feb 21, 2022
Visual analysis and diagnostic tools to facilitate machine learning model selection.

Yellowbrick Visual analysis and diagnostic tools to facilitate machine learning model selection. What is Yellowbrick? Yellowbrick is a suite of visual

District Data Labs 3.5k Feb 25, 2022
A game theoretic approach to explain the output of any machine learning model.

SHAP (SHapley Additive exPlanations) is a game theoretic approach to explain the output of any machine learning model. It connects optimal credit allo

Scott Lundberg 15.5k Feb 21, 2022
Model analysis tools for TensorFlow

TensorFlow Model Analysis TensorFlow Model Analysis (TFMA) is a library for evaluating TensorFlow models. It allows users to evaluate their models on

null 1.1k Feb 23, 2022
Tool for visualizing attention in the Transformer model (BERT, GPT-2, Albert, XLNet, RoBERTa, CTRL, etc.)

Tool for visualizing attention in the Transformer model (BERT, GPT-2, Albert, XLNet, RoBERTa, CTRL, etc.)

Jesse Vig 3.6k Feb 22, 2022
JittorVis - Visual understanding of deep learning model.

JittorVis is a deep neural network computational graph visualization library based on Jittor.

null 161 Feb 7, 2022
Quickly and easily create / train a custom DeepDream model

Dream-Creator This project aims to simplify the process of creating a custom DeepDream model by using pretrained GoogleNet models and custom image dat

null 43 Feb 16, 2022
An intuitive library to add plotting functionality to scikit-learn objects.

Welcome to Scikit-plot Single line functions for detailed visualizations The quickest and easiest way to go from analysis... ...to this. Scikit-plot i

Reiichiro Nakano 2.2k Feb 20, 2022
A library for debugging/inspecting machine learning classifiers and explaining their predictions

ELI5 ELI5 is a Python package which helps to debug machine learning classifiers and explain their predictions. It provides support for the following m

null 2.5k Feb 18, 2022
A library that implements fairness-aware machine learning algorithms

Themis ML themis-ml is a Python library built on top of pandas and sklearnthat implements fairness-aware machine learning algorithms. Fairness-aware M

Niels Bantilan 88 Feb 9, 2022
null 131 Jun 25, 2021
Lucid library adapted for PyTorch

Lucent PyTorch + Lucid = Lucent The wonderful Lucid library adapted for the wonderful PyTorch! Lucent is not affiliated with Lucid or OpenAI's Clarity

Lim Swee Kiat 435 Feb 17, 2022
FairML - is a python toolbox auditing the machine learning models for bias.

======== FairML: Auditing Black-Box Predictive Models FairML is a python toolbox auditing the machine learning models for bias. Description Predictive

Julius Adebayo 325 Feb 24, 2022
python partial dependence plot toolbox

PDPbox python partial dependence plot toolbox Motivation This repository is inspired by ICEbox. The goal is to visualize the impact of certain feature

Li Jiangchun 656 Feb 23, 2022
Python implementation of R package breakDown

pyBreakDown Python implementation of breakDown package (https://github.com/pbiecek/breakDown). Docs: https://pybreakdown.readthedocs.io. Requirements

MI^2 DataLab 40 Apr 23, 2021
⬛ Python Individual Conditional Expectation Plot Toolbox

⬛ PyCEbox Python Individual Conditional Expectation Plot Toolbox A Python implementation of individual conditional expecation plots inspired by R's IC

Austin Rochford 121 Feb 23, 2022
👋🦊 Xplique is a Python toolkit dedicated to explainability, currently based on Tensorflow.

???? Xplique is a Python toolkit dedicated to explainability, currently based on Tensorflow.

DEEL 64 Feb 11, 2022
L2X - Code for replicating the experiments in the paper Learning to Explain: An Information-Theoretic Perspective on Model Interpretation.

L2X Code for replicating the experiments in the paper Learning to Explain: An Information-Theoretic Perspective on Model Interpretation at ICML 2018,

Jianbo Chen 99 Feb 23, 2022
Code for "High-Precision Model-Agnostic Explanations" paper

Anchor This repository has code for the paper High-Precision Model-Agnostic Explanations. An anchor explanation is a rule that sufficiently “anchors”

Marco Tulio Correia Ribeiro 685 Feb 21, 2022
Collection of NLP model explanations and accompanying analysis tools

Thermostat is a large collection of NLP model explanations and accompanying analysis tools. Combines explainability methods from the captum library wi

null 88 Feb 24, 2022
TensorFlow Decision Forests (TF-DF) is a collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models.

TensorFlow Decision Forests (TF-DF) is a collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models. The library is a collection of Keras models and supports classification, regression and ranking. TF-DF is a TensorFlow wrapper around the Yggdrasil Decision Forests C++ libraries. Models trained with TF-DF are compatible with Yggdrasil Decision Forests' models, and vice versa.

null 399 Feb 25, 2022
Interpretation of T cell states using reference single-cell atlases

Interpretation of T cell states using reference single-cell atlases ProjecTILs is a computational method to project scRNA-seq data into reference sing

Cancer Systems Immunology Lab 91 Feb 18, 2022
Bot interpretation of the carbon.now.sh site

?? Source code of the @PicodeBot ?? Developer: @hoosnick Run $ git clone https://github.com/hoosnick/picodebot.git $ pip install -r requirements.txt P

Husniddin Murodov 8 Jan 24, 2022
PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

Saim Wani 2 Dec 13, 2021
Telop - Encode and decode messages using an interpretation of the telegraphic code devised by José María Mathé

telop Telop (TELégrafoÓPtico) - Utilidad para codificar y descodificar mensajes de texto empleando una interpretación del código telegráfico ideado po

Ricardo F. 2 Jan 7, 2022
A web-based analysis toolkit for the System Usability Scale providing calculation, plotting, interpretation and contextualization utility

System Usability Scale Analysis Toolkit The System Usability Scale (SUS) Analysis Toolkit is a web-based python application that provides a compilatio

Jonas Blattgerste 2 Jan 11, 2022
audioLIME: Listenable Explanations Using Source Separation

audioLIME This repository contains the Python package audioLIME, a tool for creating listenable explanations for machine learning models in music info

Institute of Computational Perception 22 Feb 27, 2022
Code for paper: Group-CAM: Group Score-Weighted Visual Explanations for Deep Convolutional Networks

Group-CAM By Zhang, Qinglong and Rao, Lu and Yang, Yubin [State Key Laboratory for Novel Software Technology at Nanjing University] This repo is the o

zhql 73 Feb 16, 2022
PyTorch implementation of Interpretable Explanations of Black Boxes by Meaningful Perturbation

PyTorch implementation of Interpretable Explanations of Black Boxes by Meaningful Perturbation The paper: https://arxiv.org/abs/1704.03296 What makes

Jacob Gildenblat 310 Feb 14, 2022