Blaze translates a subset of modified NumPy and Pandas-like syntax to databases and other computing systems. Blaze allows Python users a familiar interface to query data living in other data storage systems.
Example
We point blaze to a simple dataset in a foreign database (PostgreSQL). Instantly we see results as we would see them in a Pandas DataFrame.
>>> import blaze as bz
>>> iris = bz.Data('postgresql://localhost::iris')
>>> iris
sepal_length sepal_width petal_length petal_width species
0 5.1 3.5 1.4 0.2 Iris-setosa
1 4.9 3.0 1.4 0.2 Iris-setosa
2 4.7 3.2 1.3 0.2 Iris-setosa
3 4.6 3.1 1.5 0.2 Iris-setosa
These results occur immediately. Blaze does not pull data out of Postgres, instead it translates your Python commands into SQL (or others.)
>>> iris.species.distinct()
species
0 Iris-setosa
1 Iris-versicolor
2 Iris-virginica
>>> bz.by(iris.species, smallest=iris.petal_length.min(),
... largest=iris.petal_length.max())
species largest smallest
0 Iris-setosa 1.9 1.0
1 Iris-versicolor 5.1 3.0
2 Iris-virginica 6.9 4.5
This same example would have worked with a wide range of databases, on-disk text or binary files, or remote data.
What Blaze is not
Blaze does not perform computation. It relies on other systems like SQL, Spark, or Pandas to do the actual number crunching. It is not a replacement for any of these systems.
Blaze does not implement the entire NumPy/Pandas API, nor does it interact with libraries intended to work with NumPy/Pandas. This is the cost of using more and larger data systems.
Blaze is a good way to inspect data living in a large database, perform a small but powerful set of operations to query that data, and then transform your results into a format suitable for your favorite Python tools.
In the Abstract
Blaze separates the computations that we want to perform:
>>> accounts = Symbol('accounts', 'var * {id: int, name: string, amount: int}')
>>> deadbeats = accounts[accounts.amount < 0].name
From the representation of data
>>> L = [[1, 'Alice', 100],
... [2, 'Bob', -200],
... [3, 'Charlie', 300],
... [4, 'Denis', 400],
... [5, 'Edith', -500]]
Blaze enables users to solve data-oriented problems
>>> list(compute(deadbeats, L))
['Bob', 'Edith']
But the separation of expression from data allows us to switch between different backends.
Here we solve the same problem using Pandas instead of Pure Python.
>>> df = DataFrame(L, columns=['id', 'name', 'amount'])
>>> compute(deadbeats, df)
1 Bob
4 Edith
Name: name, dtype: object
Blaze doesn't compute these results, Blaze intelligently drives other projects to compute them instead. These projects range from simple Pure Python iterators to powerful distributed Spark clusters. Blaze is built to be extended to new systems as they evolve.
Getting Started
Blaze is available on conda or on PyPI
conda install blaze pip install blaze
Development builds are accessible
conda install blaze -c blaze pip install http://github.com/blaze/blaze --upgrade
You may want to view the docs, the tutorial, some blogposts, or the mailing list archives.
Development setup
The quickest way to install all Blaze dependencies with conda is as follows
conda install blaze spark -c blaze -c anaconda-cluster -y conda remove odo blaze blaze-core datashape -y
After running these commands, clone odo, blaze, and datashape from GitHub directly. These three projects release together. Run python setup.py develop to make development installations of each.
License
Released under BSD license. See LICENSE.txt for details.
Blaze development is sponsored by Continuum Analytics.
![BUG:TypeError: Cannot interpret 'CategoricalDtype(categories=['no', 'yes'], ordered=False)' as a data type](https://avatars.githubusercontent.com/u/4946163?v=4)
13 Oct 12, 2022
134 Dec 15, 2022
207 Jan 01, 2023
97 Dec 25, 2022
92 Dec 26, 2022
7 Dec 28, 2022
2 Nov 17, 2021
2.7k Jan 07, 2023
1 Jul 12, 2022
1.2k Jan 01, 2023
16 Nov 03, 2022
68 Dec 08, 2022
14 Aug 14, 2022
338 Nov 25, 2022
1 Feb 05, 2022
20 Nov 21, 2022
13.1k Jan 03, 2023
52 Jun 10, 2022
13 Oct 27, 2021
68 Aug 18, 2022