numpy2tfrecord
Simple helper library to convert a collection of numpy data to tfrecord, and build a tensorflow dataset from the tfrecord.
Installation
$ git clone [email protected]:yonetaniryo/numpy2tfrecord.git
$ cd numpy2tfrecord
$ pip install .
Test
$ pytest -v
How to use
Convert a collection of numpy data to tfrecord
import numpy as np
from numpy2tfrecord import Numpy2Tfrecord
converter = Numpy2Tfrecord()
x = np.arange(100).reshape(10, 10).astype(np.float32) # float array
y = np.arange(100).reshape(10, 10).astype(np.int64) # int array
a = 5 # int
b = 0.3 # float
entry = {"x": x, "y": y, "a": a, "b": b}
converter.add(entry) # add data entry
...
converter.export_to_tfrecord("test.tfrecord") # export to tfrecord
Build a tensorflow dataset from tfrecord
dataset = build_dataset_from_tfrecord("test.tfrecord") # load tfrecord and build tf.data.Dataset
5.7k Jan 9, 2023
1.3k Jan 2, 2023
1.8k Jan 8, 2023
2.6k Jan 4, 2023
92 Dec 30, 2022
34 Dec 27, 2022
3 Sep 23, 2022
102 Dec 17, 2022
1 Jan 10, 2022
120 Dec 15, 2022
109 Dec 16, 2022
5.5k Dec 29, 2022
10 Nov 03, 2022
3.7k Dec 31, 2022
1 Aug 24, 2022
48 Jan 02, 2023
4 Jan 02, 2022
147 Nov 18, 2022
230 Dec 31, 2022
16 Jan 03, 2023
56 Dec 12, 2022
2.4k Dec 28, 2022
65 Dec 20, 2022
41 Dec 10, 2022
1 Dec 10, 2021
5 Dec 20, 2022
122 Jan 04, 2023
1 Jan 13, 2022
573 Jan 05, 2023