Python package for hypergraph analysis and visualization.

Overview

HyperNetX

The HyperNetX library provides classes and methods for the analysis and visualization of complex network data. HyperNetX uses data structures designed to represent set systems containing nested data and/or multi-way relationships. The library generalizes traditional graph metrics to hypergraphs.

The current version is preliminary. We are actively testing and would be grateful for comments and suggestions. Expect changes in both class names and methods as many of the requirements demanded of the library are worked out.

HypernetX was developed by the Pacific Northwest National Laboratory for the Hypernets project as part of its High Performance Data Analytics (HPDA) program. PNNL is operated by Battelle Memorial Institute under Contract DE-AC05-76RL01830.

  • Principle Developer and Designer: Brenda Praggastis
  • Graphics Developer and Designer: Dustin Arendt
  • Principle Investigator: Emilie Purvine
  • Program Manager: Cliff Joslyn, Mark Raugas
  • Mathematics, methods, and algorithms: Sinan Aksoy, Dustin Arendt, Cliff Joslyn, Brenda Praggastis, and Emilie Purvine
  • Software support: Kyle Monson

For questions and comments you may contact the developers directly at:
[email protected]

Documentation is available at: https://pnnl.github.io/HyperNetX/

Tutorials may be run in your browser using Google Colab

Open In Colab Tutorial 1 - HNX Basics
Open In Colab Tutorial 2 - Visualization Methods
Open In Colab Tutorial 3 - LesMis Case Study
Open In Colab Tutorial 4 - LesMis Visualizations-Book Tour
Open In Colab Tutorial 5 - Homology mod2 for TriLoop Example

Installation Instructions

HyperNetX requires Python>=3.6. We recommend installation in a virtual environment.

To install in an Anaconda environment

>>> conda create -n  python=3.6
>>> source activate  

Mac Users: If you wish to build the documentation you will need the conda version of matplotlib:

>>> conda install matplotlib

To install in a virtualenv environment

>>> virtualenv --python= 

This will create a virtual environment in the specified location using the specified python executable. For example:

>>> virtualenv --python=C:\Anaconda3\python.exe hnx

This will create a virtual environment in .\hnx using the python that comes with Anaconda3.

>>> \Scripts\activate

If you are running in Windows PowerShell use =.ps1

If you are running in Windows Command Prompt use =.bat

Otherwise use =NULL (no file extension).

Once activated continue to follow the installation instructions below.

Install using Pip

For a minimal installation:

>>> pip install hypernetx

For an editable installation with access to jupyter notebooks:

>>> pip install [-e] .

To install with the tutorials:

>>> pip install -e .['tutorials']

To install with the documentation:

>>> pip install -e .['documentation']
>>> sphinx-build -b html docs/source docs/build 
## This will generate the documentation in /docs/build
## Open them in your browser with /docs/build/index.html

To install and test using pytest:

>>> pip install -e .['testing']
>>> pytest

To install the whole shabang:

>>> pip install -e .['all']

License

Released under the 3-Clause BSD license (see License.rst)

Comments
  • Not drawing the hypergraph

    Not drawing the hypergraph

    I tried the programs in the tutorial and also the code of the guy that reported the other issue and i get the following error:

    /usr/local/lib/python3.7/site-packages/hypernetx/drawing/rubber_band.py:97: FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.

    I don't know if the problem is that I use python3

    opened by ChrisGeorgakidis 10
  • Started work on support for weighted hypernetworks

    Started work on support for weighted hypernetworks

    Here is a new PR for the weighted hypernetworks code, with all the new commits in a separate branch. Here is the original comment for reference:

    I thought I would try to make it a bit easier to create a weighted hypergraph, using a similar syntax to that in networkx. Now you can add weighted edges like this:

    w = 0.019281
    G = hnx.Hypergraph()
    G.add_edge(("A", "B"), weight=w)
    

    I've also added support for calculating the weighted ~~node~~ adjacency matrices:

    w0 = 0.98279384
    w1 = 0.2309
    G = hnx.Hypergraph()
    e0 = [("A", "B"), ("B", "C")]
    e1 = [("A", "B", "C")]
    G.add_edges_from(e0, weight=w0)
    G.add_edges_from(e1, weight=w1)
    	
    print(G.adjacency_matrix())
    print(G.adjacency_matrix(weight="weight"))
    print(G.edge_adjacency_matrix(weight="weight"))
    

    ~~I have an issue with the edge adjacency matrix, because I would ordinarily calculate it with the following matrix formula: \sqrt(W) * M^T * M * \sqrt(W), but this won't currently work as edge_adjacency_matrix calls __incidence_to_adjacency, the same method as is called by adjacency_matrix. I would be grateful for your thoughts on this, I've currently left the weighted edge adjacency to throw a NotImplementedError.~~

    I have modified the code to do what I commented in the original PR. When the weighted adjacency matrix is requested, M.dot(weight_matrix) is passed into __incidence_to_adjacency so both adjacency matrix functions work.

    All of these changes play quite nicely with the existing code, but I wanted to submit them for your consideration before doing too much more.

    Many thanks. :)

    opened by jim-rafferty 6
  • Started work on support for weighted hypergraphs

    Started work on support for weighted hypergraphs

    Hi,

    I thought I would try to make it a bit easier to create a weighted hypergraph, using a similar syntax to that in networkx. Now you can add weighted edges like this:

    w = 0.019281
    G = hnx.Hypergraph()
    G.add_edge(("A", "B"), weight=w)
    

    I've also added support for calculating the weighted node adjacency matrix:

    w0 = 0.98279384
    w1 = 0.2309
    G = hnx.Hypergraph()
    e0 = [("A", "B"), ("B", "C")]
    e1 = [("A", "B", "C")]
    G.add_edges_from(e0, weight=w0)
    G.add_edges_from(e1, weight=w1)
    	
    print(G.adjacency_matrix())
    print(G.adjacency_matrix(weight="weight"))
    

    I have an issue with the edge adjacency matrix, because I would ordinarily calculate it with the following matrix formula: \sqrt(W) * M^T * M * \sqrt(W), but this won't currently work as edge_adjacency_matrix calls __incidence_to_adjacency, the same method as is called by adjacency_matrix. I would be grateful for your thoughts on this, I've currently left the weighted edge adjacency to throw a NotImplementedError.

    All of these changes play quite nicely with the existing code, but I wanted to submit them for your consideration before doing too much more.

    Many thanks. :)

    opened by jim-rafferty 6
  • Please provide support for weighted hypergraphs

    Please provide support for weighted hypergraphs

    Hi, this is a really good package :)

    It would be loads more useful if you could provide support for weighted hypergraphs. It seems to be possible to create a weighted hypergraph at the moment like this:

    import itertools
    import hypernetx as hnx
    import numpy as np
    
    labels = "ABCD"
    nodes = set(labels)
    edges = list(
        itertools.chain(
            *[list(itertools.combinations(set(labels), ii)) for ii in range(2, len(labels))]
        )
    )
    hnx_edges = hnx.EntitySet(
        "Edges", 
        [hnx.Entity(
                "".join(ee), 
                ee,
                weight=np.random.rand() 
            ) for ee in edges]
    )
    
    h = hnx.Hypergraph(hnx_edges)
    hnx.draw(h)
    print(h.edges.elements)
    

    but it's not possible to do any analysis with the weights as far as I can tell (see for example, the output of hnx.dist_stats(h) )

    Are there any plans to add this type of functionality?

    Thanks!

    opened by jim-rafferty 6
  • Import fails due to missing `decorator` package in clean install

    Import fails due to missing `decorator` package in clean install

    Installing hypernetx in a clean virtual environment fails due to a missing package: decorator.

    Step to reproduce (after initializing a new empty virtual environment):

    pip install hypernetx
    

    giving an error like this:

    >>> import hypernetx as hnx
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/__init__.py", line 7, in <module>
        from hypernetx.classes import *
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/classes/__init__.py", line 2, in <module>
        from .hypergraph import Hypergraph
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/classes/hypergraph.py", line 13, in <module>
        from hypernetx.classes.staticentity import StaticEntity, StaticEntitySet
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/classes/staticentity.py", line 10, in <module>
        from hypernetx.utils import (
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/utils/__init__.py", line 8, in <module>
        from .decorators import not_implemented_for
      File "/tmp/hnx-version/venv/lib/python3.8/site-packages/hypernetx/utils/decorators.py", line 4, in <module>
        from decorator import decorator
    ModuleNotFoundError: No module named 'decorator'
    

    Of course, installing the decorator package manually fixes the issue and hypernetx imports fine. I think the decorator package should just be added to the install_requires entry in setup.cfg and setup.py since this package is required to even load the package at all.

    Tested using Python versions 3.8 and 3.9, using hypernetx 1.2.2 (on Ubuntu).

    opened by tmthyln 3
  • Dependencies are not correct

    Dependencies are not correct

    For pip install hypernetx, the packages igraph and celluloid are not installed by default. One has to do it manually. This is also the reason why the tutorial fails (#73). What about adding them to setup.py?

    opened by michaeldorner 3
  • Find all hyperedges for a node

    Find all hyperedges for a node

    Hi, Maybe I am just blind (and/or stupid), but how can I identify (easily) all hyperedges of a given node n in a (static) hypergraph H in HyperNetX?

    The solution I came up is:

    [edge for edge, nodes in H.incidence_dict.items() if n in nodes]

    But probably you had another, more performant solution in mind? Thank you!

    opened by michaeldorner 2
  • Import error on module hypernetx.utils.toys

    Import error on module hypernetx.utils.toys

    Hello, I have the following error when I try to import hypernetx on Colab (with your tutorials) and also on my conda env:

    ModuleNotFoundError                       Traceback (most recent call last)
    <ipython-input-3-b7c66bb90caa> in <module>()
          3 import matplotlib.pyplot as plt
          4 import networkx as nx
    ----> 5 import hypernetx as hnx
          6 
    
    4 frames
    /usr/local/lib/python3.7/dist-packages/hypernetx/__init__.py in <module>()
          5 )
          6 from hypernetx.read_write import to_pickle, load_from_pickle
    ----> 7 from hypernetx.classes import *
          8 from hypernetx.reports import *
          9 from hypernetx.drawing import *
    
    /usr/local/lib/python3.7/dist-packages/hypernetx/classes/__init__.py in <module>()
          1 from .entity import Entity, EntitySet
    ----> 2 from .hypergraph import Hypergraph
          3 from .staticentity import StaticEntity, StaticEntitySet
    
    /usr/local/lib/python3.7/dist-packages/hypernetx/classes/hypergraph.py in <module>()
         11 from collections import OrderedDict, defaultdict
         12 from hypernetx.classes.entity import Entity, EntitySet
    ---> 13 from hypernetx.classes.staticentity import StaticEntity, StaticEntitySet
         14 from hypernetx.exception import HyperNetXError
         15 from hypernetx.utils.decorators import not_implemented_for
    
    /usr/local/lib/python3.7/dist-packages/hypernetx/classes/staticentity.py in <module>()
          8 from hypernetx.exception import HyperNetXError
          9 from hypernetx.classes.entity import Entity, EntitySet
    ---> 10 from hypernetx.utils import HNXCount, DefaultOrderedDict, remove_row_duplicates
         11 from scipy.sparse import coo_matrix, csr_matrix, issparse
         12 import itertools as it
    
    /usr/local/lib/python3.7/dist-packages/hypernetx/utils/__init__.py in <module>()
          2 from .decorators import not_implemented_for
          3 # from .toys import HarryPotter, LesMis, lesmis_hypergraph_from_df, book_tour, TransmissionProblem
    ----> 4 from .toys import *
    
    ModuleNotFoundError: No module named 'hypernetx.utils.toys'
    

    I'm doing something wrong?

    opened by RemiG3 2
  • add_node_to_edge does not update the node's membership

    add_node_to_edge does not update the node's membership

    When adding an existing node to an existing edge, the memberships property of the node stored in _nodes is not updated

    Code to reproduce the issue:

    system = hnx.Hypergraph()
    edge0 = hnx.Entity('edge0')
    edge1 = hnx.Entity('edge1')
    node0 = hnx.Entity('node0')
    node1 = hnx.Entity('node1')
    
    system.add_edge(edge0)
    system.add_edge(edge1)
    system.add_node_to_edge(node0,edge0)
    system.add_node_to_edge(node0,edge1)
    system.add_node_to_edge(node1,edge1)
    
    #Memberships seen from the nodes stored within _edges
    print("Memberships seen from the nodes stored within _edges")
    for el in system.edges:
        for edge in system.edges[el]:
            print(edge,system.edges[el][edge].memberships)
            
    #Memberships seen from the nodes stored within _nodes
    print("Memberships seen from the nodes stored within _nodes")
    for el in system.nodes:
        print(el,system.nodes[el].memberships)
    
    hnx.drawing.rubber_band.draw(system)
    
    opened by dncolomer 2
  • Overlapping node labels

    Overlapping node labels

    When drawing hypergraphs using the rubber band layout, node labels often overlap. This is especially prevalent when using longer strings as node labels; however, this problem also often occurs with very short labels as well. Due to the overlap, many labels are unreadable. Are there any settings within HyperNetX or Matplotlib to correct this? If not, are there any known workarounds? Thanks!

    opened by epideveloper 2
  • Can I adjust how close hyperedges can be drawn in proximity to nodes?

    Can I adjust how close hyperedges can be drawn in proximity to nodes?

    I made the following diagram using Hypernetworkx's draw command.

    image

    The node fst_score node appears really close to the 233 hyperedge, making it hard to distinguish whether the node is a member of the hyperedge. Is there a setting an argument somewhere that would allow me to ensure that the hyperedges maintain a minimal distance from the nodes?

    opened by galenseilis 1
  • Same edge name ?

    Same edge name ?

    Is it possible for edges with the same name but different nodes to coexist. Something like this :

    {
      "is" : ["red","apple","rotten"],
      "is" : ["black","pen"]
    }
    

    How do you achieve this ?

    opened by vsraptor 1
  • Comparison between weighted and unweighted hypergraphs in clustering

    Comparison between weighted and unweighted hypergraphs in clustering

    Hello, thank you for sharing this excellent library.

    I want to compare the performance of clustering on weighted and unweighted hypergraphs as described in K. Hayashi, S. Aksoy, C. Park, H. Park, "Hypergraph random walks, Laplacians, and clustering". I tried to do this based on Tutorial 11. However, I think some modifications need. h = hnx.Hypergraph(hnx.StaticEntitySet(data=data, weights=w)) I think this code does not make a weighted hypergraph. Instead, this should be revised as below. h = hnx.Hypergraph(hnx.StaticEntitySet(data=data, weights=w), weights=w)

    Additionally, I cannot get the satisfying result that weighted hypergraphs perform better than unweighted ones. Thus, I want to ask whether the below code is a correct way to make clustering on an unweighted hypergraph and evaluate clustering algorithms by NMI scores.

    # get data
    categories = all_categories[[1,15]]
    twenty_train = fetch_20newsgroups(subset='test', categories=categories, shuffle=True, random_state=42)
    doc_types=dict()
    for i,x in enumerate(twenty_train.filenames):
        doc_types[i]=x.split('/')[-2]
    tfidf_vect = TfidfVectorizer()
    X_tfidf = tfidf_vect.fit_transform(twenty_train.data)
    
    # construct hypergraph
    mat = coo_matrix(X_tfidf)
    edges = mat.col
    nodes = mat.row
    data = np.array([edges,nodes]).T
    weights = mat.data
    # clustering on weighted hypergraph
    h = hnx.Hypergraph(hnx.StaticEntitySet(data=data,weights=weights),weights=weights)
    clusters=hnx.spec_clus(h,num_clus,weights=True)
    # clustering on unweighted hypergraph
    uwh =  hnx.Hypergraph(hnx.StaticEntitySet(data=data,weights=None))
    uw_clusters = hnx.spec_clus(uwh,num_clus,weights=False)
    
    # construct clustering label list
    categoryindexing = {}
    _labels = {} # answer
    for i in range(X_tfidf.shape[0]):
        if doc_types[i] not in categoryindexing:
            categoryindexing[doc_types[i]] = len(categoryindexing)
        ci = categoryindexing[doc_types[i]]
        _labels[i] = ci
    labels = [_labels[i] for i in range(X_tfidf.shape[0])]
    _w_pred = {} #  labels from weighted hypergraph
    for i in clusters:
        for v in clusters[i]:
            _w_pred[v] = i
    w_pred = [_w_pred[i] for i in range(X_tfidf.shape[0])]
    _uw_pred = {} # labels from unweighted hypergraph
    for i in uw_clusters:
        for v in uw_clusters[i]:
            _uw_pred[v] = i
    uw_pred = [_uw_pred[i] for i in range(X_tfidf.shape[0])]
    
    # evaluation on NMI score
    from sklearn.metrics.cluster import normalized_mutual_info_score as NMI_SCORE
    score = NMI_SCORE(labels, w_pred)
    print(score) # 0.73
    score = NMI_SCORE(labels, uw_pred)
    print(score) # 0.78
    
    opened by young917 3
  • Modularity function returns error: AttributeError: 'Entity' object has no attribute 'strength'

    Modularity function returns error: AttributeError: 'Entity' object has no attribute 'strength'

    When I call print('qH =',hmod.modularity(H, K, strict)) it returns AttributeError: 'Entity' object has no attribute 'strength'

    The full code:

    import igraph as ig
    import hypernetx as hnx
    import hypernetx.algorithms.hypergraph_modularity as hmod
    
    H = hnx.Hypergraph(lookup)
    H = hmod.precompute_attributes(H)
    #%%
    K = hmod.kumar(H)
    #%%
    strict = hmod.strict
    ## Compute qH
    print('qH =',hmod.modularity(H, K, strict))
    

    Lookup is a dictionary of the form: {key1 : (hedge1, hedge2...), key2 : (hedge1, hedge3...)}

    opened by thosvarley 3
  • Nodes in a Hypergarph via H.nodes

    Nodes in a Hypergarph via H.nodes

    I tried to generate a hypergraph using node neighborhood information from a matrix K of size |N| x |N|. Using the simple graph edges list, if the length (distance) of two given nodes is less than or equal to k, then I put 1 in the K matrix, 0 otherwise. 1 denotes that the two given nodes are neighbors, and 0 denotes that nodes are not neighbors. After generating the K matrix, I used it to represent hyperedges where each row of K represents the neighborhood of a particular node. (a hyperedge).

    The problem is, after generating a hypergraph, it shows the correct number of hyperedges using the H.edges method, however, it gives the wrong number of nodes using the H.nodes method. The total nodes are very less than the total nodes given by the H.nodes method,

    Also, I tried the example graph given in the documentation of the library. It has 13 nodes and 8 hyperedges. H.nodes and H.edges methods show the correct number of nodes and edges for this graph.

    Can't I use the defined K matrix for hypergraph generation?

    opened by khizer-hayat 1
Releases(v1.2.5)
Owner
Pacific Northwest National Laboratory
Pacific Northwest National Laboratory
Chem: collection of mostly python code for molecular visualization, QM/MM, FEP, etc

chem: collection of mostly python code for molecular visualization, QM/MM, FEP,

5 Sep 02, 2022
Create HTML profiling reports from pandas DataFrame objects

Pandas Profiling Documentation | Slack | Stack Overflow Generates profile reports from a pandas DataFrame. The pandas df.describe() function is great

10k Jan 01, 2023
Automatically generate GitHub activity!

Commit Bot Automatically generate GitHub activity! We've all wanted to be the developer that commits every day, but that requires a lot of work. Let's

Ricky 4 Jun 07, 2022
Insert SVGs into matplotlib

Insert SVGs into matplotlib

Andrew White 35 Dec 29, 2022
This is a Cross-Platform Plot Manager for Chia Plotting that is simple, easy-to-use, and reliable.

Swar's Chia Plot Manager A plot manager for Chia plotting: https://www.chia.net/ Development Version: v0.0.1 This is a cross-platform Chia Plot Manage

Swar Patel 1.3k Dec 13, 2022
Lightspin AWS IAM Vulnerability Scanner

Red-Shadow Lightspin AWS IAM Vulnerability Scanner Description Scan your AWS IAM Configuration for shadow admins in AWS IAM based on misconfigured den

Lightspin 90 Dec 14, 2022
Custom Plotly Dash components based on Mantine React Components library

Dash Mantine Components Dash Mantine Components is a Dash component library based on Mantine React Components Library. It makes it easier to create go

Snehil Vijay 239 Jan 08, 2023
Jupyter Notebook extension leveraging pandas DataFrames by integrating DataTables and ChartJS.

Jupyter DataTables Jupyter Notebook extension to leverage pandas DataFrames by integrating DataTables JS. About Data scientists and in fact many devel

Marek Čermák 142 Dec 28, 2022
NumPy and Pandas interface to Big Data

Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar inte

Blaze 3.1k Jan 01, 2023
Browse Dash docsets inside emacs

Helm Dash What's it This package uses Dash docsets inside emacs to browse documentation. Here's an article explaining the basic usage of it. It doesn'

504 Dec 15, 2022
Process dataframe in a easily way.

Popanda Written by Shengxuan Wang at OSU. Used for processing dataframe, especially for machine learning. The name is from "Po" in the movie Kung Fu P

ShawnWang 1 Dec 24, 2021
python partial dependence plot toolbox

PDPbox python partial dependence plot toolbox Motivation This repository is inspired by ICEbox. The goal is to visualize the impact of certain feature

Li Jiangchun 723 Jan 07, 2023
An animation engine for explanatory math videos

Powered By: An animation engine for explanatory math videos Hi there, I'm Zheer 👋 I'm a Software Engineer and student!! 🌱 I’m currently learning eve

Zaheer ud Din Faiz 2 Nov 04, 2021
Main repository for Vispy

VisPy: interactive scientific visualization in Python Main website: http://vispy.org VisPy is a high-performance interactive 2D/3D data visualization

vispy 3k Jan 03, 2023
Make your BSC transaction simple.

bsc_trade_history Make your BSC transaction simple. 中文ReadMe Background: inspired by debank ,Practice my hands on this small project Blog:Crypto-BscTr

foolisheddy 7 Jul 06, 2022
The windML framework provides an easy-to-use access to wind data sources within the Python world, building upon numpy, scipy, sklearn, and matplotlib. Renewable Wind Energy, Forecasting, Prediction

windml Build status : The importance of wind in smart grids with a large number of renewable energy resources is increasing. With the growing infrastr

Computational Intelligence Group 125 Dec 24, 2022
100 Days of Code The Complete Python Pro Bootcamp for 2022

100-Day-With-Python 100 Days of Code - The Complete Python Pro Bootcamp for 2022. In this course, I spend with python language over 100 days, and I up

Rajdip Das 8 Jun 22, 2022
Visualise Ansible execution time across playbooks, tasks, and hosts.

ansible-trace Visualise where time is spent in your Ansible playbooks: what tasks, and what hosts, so you can find where to optimise and decrease play

Mark Hansen 81 Dec 15, 2022
Learn Basic to advanced level Data visualisation techniques from this Repository

Data visualisation Hey, You can learn Basic to advanced level Data visualisation techniques from this Repository. Data visualization is the graphic re

Shashank dwivedi 16 Jan 03, 2023
Flipper Zero documentation repo

Flipper Zero Docs Participation To fix a bug or add something new to this repository, you need to open a pull-request. Also, on every page of the site

Flipper Zero (All Repositories will be public soon) 114 Dec 30, 2022