An easier way to build neural search on the cloud

Overview

Jina banner

An easier way to build neural search on the cloud

Jina Python 3.7 3.8 3.9 PyPI Docker Image Version (latest semver) CI CD codecov

Jina is a deep learning-powered search framework for building cross-/multi-modal search systems (e.g. text, images, video, audio) on the cloud.

โฑ๏ธ Time Saver - The design pattern of neural search systems, from zero to a production-ready system in minutes.

๐Ÿฑ Full-Stack Ownership - Keep an end-to-end stack ownership of your solution, avoid the integration pitfalls with fragmented, multi-vendor, generic legacy tools.

๐ŸŒŒ Universal Search - Large-scale indexing and querying of unstructured data: video, image, long/short text, music, source code, etc.

๐Ÿง  First-Class AI Models - First-class support for state-of-the-art AI models, easily usable and extendable with a Pythonic interface.

๐ŸŒฉ๏ธ Fast & Cloud Ready - Decentralized architecture from day one. Scalabe & cloud-native by design: enjoy containerizing, distributing, sharding, async, REST/gRPC/WebSocket.

โค๏ธ Made with Love - Never compromise on quality, actively maintained by a passionate full-time, venture-backed team.


Docs โ€ข Hello World โ€ข Quick Start โ€ข Learn โ€ข Examples โ€ข Contribute โ€ข Jobs โ€ข Website โ€ข Slack

Installation

๐Ÿ“ฆ
x86/64,arm/v6,v7,v8 (Apple M1)
On Linux/macOS & Python 3.7/3.8/3.9 Docker Users
Standard pip install -U jina docker run jinaai/jina:latest
Daemon pip install -U "jina[daemon]" docker run --network=host jinaai/jina:latest-daemon
With Extras pip install -U "jina[devel]" docker run jinaai/jina:latest-devel
Dev/Pre-Release pip install --pre jina docker run jinaai/jina:master

Version identifiers are explained here. To install Jina with extra dependencies please refer to the docs. Jina can run on Windows Subsystem for Linux. We welcome the community to help us with native Windows support.

Jina "Hello, World!" ๐Ÿ‘‹ ๐ŸŒ

Just starting out? Try Jina's "Hello, World" - a simple image neural search demo for Fashion-MNIST. No extra dependencies needed, simply run:

jina hello-world  # more options in --help

...or even easier for Docker users, no install required:

docker run -v "$(pwd)/j:/j" jinaai/jina hello-world --workdir /j && open j/hello-world.html
# replace "open" with "xdg-open" on Linux
Click here to see console output

hello world console output

This downloads the Fashion-MNIST training and test dataset and tells Jina to index 60,000 images from the training set. Then it randomly samples images from the test set as queries and asks Jina to retrieve relevant results. The whole process takes about 1 minute, and after running opens a webpage and shows results:

Covid-19 Chatbot

For NLP engineers, we provide a simple chatbot demo for answering Covid-19 questions. You will need PyTorch and Transformers, which can be installed along with Jina:

pip install "jina[torch,transformers]"
jina hello-world-chatbot

This downloads CovidQA dataset and tells Jina to index 418 question-answer pairs with DistilBERT. The index process takes about 1 minute on CPU. Then it opens a webpage where you can input questions and ask Jina.

Get Started

๐Ÿฅš CRUD Functions โ€ข Document โ€ข Flow
๐Ÿฃ Feed Data โ€ข Fetch Result โ€ข Add Logic โ€ข Inter & Intra Parallelism โ€ข Decentralize โ€ข Asynchronous
๐Ÿฅ Customize Encoder โ€ข Test Encoder โ€ข Parallelism & Batching โ€ข Add Data Indexer โ€ข Compose Flow from YAML โ€ข Search โ€ข Evaluation โ€ข REST Interface

๐Ÿฅš Fundamental

CRUD Functions

First we look at basic CRUD operations. In Jina, CRUD corresponds to four functions: index (create), search (read), update, and delete. With Documents below as an example:

import numpy as np
from jina import Document
docs = [Document(id='๐Ÿฒ', embedding=np.array([0, 0]), tags={'guardian': 'Azure Dragon', 'position': 'East'}),
        Document(id='๐Ÿฆ', embedding=np.array([1, 0]), tags={'guardian': 'Vermilion Bird', 'position': 'South'}),
        Document(id='๐Ÿข', embedding=np.array([0, 1]), tags={'guardian': 'Black Tortoise', 'position': 'North'}),
        Document(id='๐Ÿฏ', embedding=np.array([1, 1]), tags={'guardian': 'White Tiger', 'position': 'West'})]

Let's build a Flow with a simple indexer:

from jina import Flow
f = Flow().add(uses='_index')

Document and Flow are basic concepts in Jina, which will be explained later. _index is a built-in embedding + structured storage that one can use out of the box.

Index
# save four docs (both embedding and structured info) into storage
with f:
    f.index(docs, on_done=print)
Search
# retrieve top-3 neighbours of ๐Ÿฒ, this print ๐Ÿฒ๐Ÿฆ๐Ÿข with score 0, 1, 1 respectively
with f:
    f.search(docs[0], top_k=3, on_done=lambda x: print(x.docs[0].matches))
{"id": "๐Ÿฒ", "tags": {"guardian": "Azure Dragon", "position": "East"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"opName": "NumpyIndexer", "refId": "๐Ÿฒ"}, "adjacency": 1}
{"id": "๐Ÿฆ", "tags": {"position": "South", "guardian": "Vermilion Bird"}, "embedding": {"dense": {"buffer": "AQAAAAAAAAAAAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"value": 1.0, "opName": "NumpyIndexer", "refId": "๐Ÿฒ"}, "adjacency": 1}
{"id": "๐Ÿข", "tags": {"guardian": "Black Tortoise", "position": "North"}, "embedding": {"dense": {"buffer": "AAAAAAAAAAABAAAAAAAAAA==", "shape": [2], "dtype": "<i8"}}, "score": {"value": 1.0, "opName": "NumpyIndexer", "refId": "๐Ÿฒ"}, "adjacency": 1}
Update
# update ๐Ÿฒ embedding in the storage
docs[0].embedding = np.array([1, 1])
with f:
    f.update(docs[0])
Delete
# remove ๐Ÿฆ๐Ÿฒ Documents from the storage
with f:
    f.delete(['๐Ÿฆ', '๐Ÿฒ'])

Document

Document is Jina's primitive data type. It can contain text, image, array, embedding, URI, and accompanied by rich meta information. To construct a Document, one can use:

import numpy
from jina import Document

doc1 = Document(content=text_from_file, mime_type='text/x-python')  # a text document contains python code
doc2 = Document(content=numpy.random.random([10, 10]))  # a ndarray document

Document can be recurred both vertically and horizontally to have nested documents and matched documents. To better see the recursive structure of a document, one can use .plot() function. If you are using JupyterLab/Notebook, all Document objects will be auto-rendered.

import numpy
from jina import Document

d0 = Document(id='๐Ÿฒ', embedding=np.array([0, 0]))
d1 = Document(id='๐Ÿฆ', embedding=np.array([1, 0]))
d2 = Document(id='๐Ÿข', embedding=np.array([0, 1]))
d3 = Document(id='๐Ÿฏ', embedding=np.array([1, 1]))

d0.chunks.append(d1)
d0.chunks[0].chunks.append(d2)
d0.matches.append(d3)

d0.plot()  # simply `d0` on JupyterLab
Click here to see more about MultimodalDocument

MultimodalDocument

A MultimodalDocument is a document composed of multiple Document from different modalities (e.g. text, image, audio).

Jina provides multiple ways to build a multimodal Document. For example, one can provide the modality names and the content in a dict:

from jina import MultimodalDocument
document = MultimodalDocument(modality_content_map={
    'title': 'my holiday picture',
    'description': 'the family having fun on the beach',
    'image': PIL.Image.open('path/to/image.jpg')
})

One can also compose a MultimodalDocument from multiple Document directly:

from jina.types import Document, MultimodalDocument

doc_title = Document(content='my holiday picture', modality='title')
doc_desc = Document(content='the family having fun on the beach', modality='description')
doc_img = Document(content=PIL.Image.open('path/to/image.jpg'), modality='image')
doc_img.tags['date'] = '10/08/2019'

document = MultimodalDocument(chunks=[doc_title, doc_description, doc_img])
Fusion Embeddings from Different Modalities

To extract fusion embeddings from different modalities Jina provides BaseMultiModalEncoder abstract class, which has a unqiue encode interface.

def encode(self, *data: 'numpy.ndarray', **kwargs) -> 'numpy.ndarray':
    ...

MultimodalDriver provides data to the MultimodalDocument in the correct expected order. In this example below, image embedding is passed to the endoder as the first argument, and text as the second.

!MyMultimodalEncoder
with:
  positional_modality: ['image', 'text']
requests:
  on:
    [IndexRequest, SearchRequest]:
      - !MultiModalDriver {}

Interested readers can refer to jina-ai/example: how to build a multimodal search engine for image retrieval using TIRG (Composing Text and Image for Image Retrieval) for the usage of MultimodalDriver and BaseMultiModalEncoder in practice.

Flow

Jina provides a high-level Flow API to simplify building CRUD workflows. To create a new Flow:

from jina import Flow
f = Flow().add()

This creates a simple Flow with one Pod. You can chain multiple .add()s in a single Flow.

To visualize the Flow, simply chain it with .plot('my-flow.svg'). If you are using a Jupyter notebook, the Flow object will be displayed inline without plot.

Gateway is the entrypoint of the Flow.

Get the vibe? Now we are talking! Let's learn more about the basic concepts and features in Jina.


๐Ÿฅš CRUD Functions โ€ข Document โ€ข Flow
๐Ÿฃ Feed Data โ€ข Fetch Result โ€ข Add Logic โ€ข Inter & Intra Parallelism โ€ข Decentralize โ€ข Asynchronous
๐Ÿฅ Customize Encoder โ€ข Test Encoder โ€ข Parallelism & Batching โ€ข Add Data Indexer โ€ข Compose Flow from YAML โ€ข Search โ€ข Evaluation โ€ข REST Interface

๐Ÿฃ Basic

Feed Data

To use a Flow, open it via with context manager, like you would open a file in Python. Now let's create some empty document and index it:

from jina import Document

with Flow().add() as f:
    f.index((Document() for _ in range(10)))

Flow supports CRUD operations: index, search, update, delete. Besides, it also provides sugary syntax on ndarray, csv, ndjson and arbitrary files.

Input Example on index/search Explain
numpy.ndarray
with f:
  f.index_ndarray(numpy.random.random([4,2]))

Input four Document, each document.blob is a ndarray([2])

CSV
with f, open('index.csv') as fp:
  f.index_csv(fp1, field_resolver={'pic_url': 'uri'})

Each line in the index.csv is constructed as Document, CSV's field pic_url is mapped to document.uri.

JSON Lines/ndjson/LDJSON
with f, open('index.ndjson') as fp:
  f.index_ndjson(fp1, field_resolver={'question_id': 'id'})

Each line in index.ndjson is constructed as Document, JSON's field question_id is mapped to document.id.

Files with wildcard
with f:
  f.index_files(['/tmp/*.mp4', '/tmp/*.pdf'])

Each file captured is constructed as a Document, whose content (text, blob, buffer) is auto-guessed & filled.

Fetch Result

Once a request is done, callback functions are fired. Jina Flow implements Promise-like interface, you can add callback functions on_done, on_error, on_always to hook different events. In the example below, our Flow passes the message then prints the result when successful. If something wrong, it beeps. Finally, the result is written to output.txt.

def beep(*args):
    # make a beep sound
    import os
    os.system('echo -n "\a";')

with Flow().add() as f, open('output.txt', 'w') as fp:
    f.index(numpy.random.random([4, 5, 2]),
            on_done=print, on_error=beep, on_always=lambda x: fp.write(x.json()))

Add Logic

To add logic to the Flow, use the uses parameter to attach a Pod with an Executor. uses accepts multiple value types including class name, Docker image, (inline) YAML or built-in shortcut.

f = (Flow().add(uses='MyBertEncoder')  # class name of a Jina Executor
           .add(uses='docker://jinahub/pod.encoder.dummy_mwu_encoder:0.0.6-0.9.3')  # the image name
           .add(uses='myencoder.yml')  # YAML serialization of a Jina Executor
           .add(uses='!WaveletTransformer | {freq: 20}')  # inline YAML config
           .add(uses='_pass')  # built-in shortcut executor
           .add(uses={'__cls': 'MyBertEncoder', 'with': {'param': 1.23}}))  # dict config object with __cls keyword

The power of Jina lies in its decentralized architecture: each add creates a new Pod, and these Pods can be run as a local thread/process, a remote process, inside a Docker container, or even inside a remote Docker container.

Inter & Intra Parallelism

Chaining .add()s creates a sequential Flow. For parallelism, use the needs parameter:

f = (Flow().add(name='p1', needs='gateway')
           .add(name='p2', needs='gateway')
           .add(name='p3', needs='gateway')
           .needs(['p1','p2', 'p3'], name='r1').plot())

p1, p2, p3 now subscribe to Gateway and conduct their work in parallel. The last .needs() blocks all Pods until they finish their work. Note: parallelism can also be performed inside a Pod using parallel:

f = (Flow().add(name='p1', needs='gateway')
           .add(name='p2', needs='gateway')
           .add(name='p3', parallel=3)
           .needs(['p1','p3'], name='r1').plot())

Decentralized Flow

A Flow does not have to be local-only, one can put any Pod to remote(s). In the example below, with the host keyword gpu-pod is put to a remote machine for parallelization, whereas other pods stay local. Extra file dependencies that need to be uploaded are specified via the upload_files keyword.

123.456.78.9
# have docker installed
docker run --name=jinad --network=host -v /var/run/docker.sock:/var/run/docker.sock jinaai/jina:latest-daemon --port-expose 8000
# to stop it
docker rm -f jinad
Local
import numpy as np
from jina import Flow

f = (Flow()
     .add()
     .add(name='gpu_pod',
          uses='mwu_encoder.yml',
          host='123.456.78.9:8000',
          parallel=2,
          upload_files=['mwu_encoder.py'])
     .add())

with f:
    f.index_ndarray(np.random.random([10, 100]), output=print)

We provide a demo server on cloud.jina.ai:8000, give the following snippet a try!

from jina import Flow

with Flow().add().add(host='cloud.jina.ai:8000') as f:
    f.index(['hello', 'world'])

Asynchronous Flow

Synchronous from outside, Jina runs asynchronously underneath: it manages the eventloop(s) for scheduling the jobs. If the user wants more control over the eventloop, then AsyncFlow comes to use.

Unlike Flow, the CRUD of AsyncFlow accepts input & output functions as async generator. This is useful when your data sources involve other asynchronous libraries (e.g. motor for MongoDB):

from jina import AsyncFlow

async def input_fn():
    for _ in range(10):
        yield Document()
        await asyncio.sleep(0.1)

with AsyncFlow().add() as f:
    async for resp in f.index(input_fn):
        print(resp)

AsyncFlow is particular useful when Jina is using as part of the integration, where another heavy-lifting job is running concurrently:

async def run_async_flow_5s():  # WaitDriver pause 5s makes total roundtrip ~5s
    with AsyncFlow().add(uses='- !WaitDriver {}') as f:
        async for resp in f.index_ndarray(numpy.random.random([5, 4])):
            print(resp)

async def heavylifting():  # total roundtrip takes ~5s
    print('heavylifting other io-bound jobs, e.g. download, upload, file io')
    await asyncio.sleep(5)
    print('heavylifting done after 5s')

async def concurrent_main():  # about 5s; but some dispatch cost, can't be just 5s, usually at <7s
    await asyncio.gather(run_async_flow_5s(), heavylifting())

if __name__ == '__main__':
    asyncio.run(concurrent_main())

AsyncFlow is very useful when using Jina inside the Jupyter Notebook. As Jupyter/ipython already manages an eventloop and thanks to autoawait, AsyncFlow can run out-of-the-box in Jupyter.

That's all you need to know for understanding the magic behind hello-world. Now let's dive into it!


๐Ÿฅš CRUD Functions โ€ข Document โ€ข Flow
๐Ÿฃ Feed Data โ€ข Fetch Result โ€ข Add Logic โ€ข Inter & Intra Parallelism โ€ข Decentralize โ€ข Asynchronous
๐Ÿฅ Customize Encoder โ€ข Test Encoder โ€ข Parallelism & Batching โ€ข Add Data Indexer โ€ข Compose Flow from YAML โ€ข Search โ€ข Evaluation โ€ข REST Interface

๐Ÿฅ Breakdown of hello-world

Customize Encoder

Let's first build a naive image encoder that embeds images into vectors using an orthogonal projection. To do this, we simply inherit from BaseImageEncoder: a base class from the jina.executors.encoders module. We then override its __init__() and encode() methods.

import numpy as np
from jina.executors.encoders import BaseImageEncoder

class MyEncoder(BaseImageEncoder):

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        np.random.seed(1337)
        H = np.random.rand(784, 64)
        u, s, vh = np.linalg.svd(H, full_matrices=False)
        self.oth_mat = u @ vh

    def encode(self, data: 'np.ndarray', *args, **kwargs):
        return (data.reshape([-1, 784]) / 255) @ self.oth_mat

Jina provides a family of Executor classes, which summarize frequently-used algorithmic components in neural search. This family consists of encoders, indexers, crafters, evaluators, and classifiers, each with a well-designed interface. You can find the list of all 107 built-in executors here. If they don't meet your needs, inheriting from one of them is the easiest way to bootstrap your own Executor. Simply use our Jina Hub CLI:

pip install jina[hub] && jina hub new

Test Encoder in Flow

Let's test our encoder in the Flow with some synthetic data:

def validate(req):
    assert len(req.docs) == 100
    assert NdArray(req.docs[0].embedding).value.shape == (64,)

f = Flow().add(uses='MyEncoder')

with f:
    f.index_ndarray(numpy.random.random([100, 28, 28]), on_done=validate)

All good! Now our validate function confirms that all one hundred 28x28 synthetic images have been embedded into 100x64 vectors.

Parallelism & Batching

By setting a larger input, you can play with batch_size and parallel:

f = Flow().add(uses='MyEncoder', parallel=10)

with f:
    f.index_ndarray(numpy.random.random([60000, 28, 28]), batch_size=1024)

Add Data Indexer

Now we need to add an indexer to store all the embeddings and the image for later retrieval. Jina provides a simple numpy-powered vector indexer NumpyIndexer, and a key-value indexer BinaryPbIndexer. We can combine them in a single YAML file:

!CompoundIndexer
components:
  - !NumpyIndexer
    with:
      index_filename: vec.gz
  - !BinaryPbIndexer
    with:
      index_filename: chunk.gz
metas:
  workspace: ./
  • ! tags a structure with a class name
  • with defines arguments for initializing this class object.

Essentially, the above YAML config is equivalent to the following Python code:

from jina.executors.indexers.vector import NumpyIndexer
from jina.executors.indexers.keyvalue import BinaryPbIndexer
from jina.executors.indexers import CompoundIndexer

a = NumpyIndexer(index_filename='vec.gz')
b = BinaryPbIndexer(index_filename='vec.gz')
c = CompoundIndexer()
c.components = lambda: [a, b]

Compose Flow from YAML

Now let's add our indexer YAML file to the Flow with .add(uses=). Let's also add two shards to the indexer to improve its scalability:

f = Flow().add(uses='MyEncoder', parallel=2).add(uses='myindexer.yml', shards=2).plot()

When you have many arguments, constructing a Flow in Python can get cumbersome. In that case, you can simply move all arguments into one flow.yml:

!Flow
version: '1.0'
pods:
  - name: encode
    uses: MyEncoder
    parallel: 2
  - name:index
    uses: myindexer.yml
    shards: 2

And then load it in Python:

f = Flow.load_config('flow.yml')

Search

Querying a Flow is similar to what we did with indexing. Simply load the query Flow and switch from f.index to f.search. Say you want to retrieve the top 50 documents that are similar to your query and then plot them in HTML:

f = Flow.load_config('flows/query.yml')
with f:
    f.search_ndarray(numpy.random.random([10, 28, 28]), shuffle=True, on_done=plot_in_html, top_k=50)

Evaluation

To compute precision recall on the retrieved result, you can add _eval_pr, a built-in evaluator for computing precision & recall.

f = (Flow().add(...)
           .add(uses='_eval_pr'))

You can construct an iterator of query and groundtruth pairs and feed to the flow f, via:

from jina import Document

def query_generator():
    for _ in range(10):
        q = Document()
        # now construct expect matches as groundtruth
        gt = Document(q, copy=True)  # make sure 'gt' is identical to 'q'
        gt.matches.append(...)
        yield q, gt

f.search(query_iterator, ...)

REST Interface

In practice, the query Flow and the client (i.e. data sender) are often physically separated. Moreover, the client may prefer to use a REST API rather than gRPC when querying. You can set port_expose to a public port and turn on REST support with restful=True:

f = Flow(port_expose=45678, restful=True)

with f:
    f.block()

That is the essence behind jina hello-world. It is merely a taste of what Jina can do. Weโ€™re really excited to see what you do with Jina! You can easily create a Jina project from templates with one terminal command:

pip install jina[hub] && jina hub new --type app

This creates a Python entrypoint, YAML configs and a Dockerfile. You can start from there.

Learn

Jina 101 Concept Illustration Book, Copyright by Jina AI Limited   

Jina 101: First Things to Learn About Jina

  English โ€ข ๆ—ฅๆœฌ่ชž โ€ข Franรงais โ€ข Portuguรชs โ€ข Deutsch โ€ข ะ ัƒััะบะธะน ัะทั‹ะบ โ€ข ไธญๆ–‡ โ€ข ุนุฑุจูŠุฉ

Examples (View all)

Example code to build your own projects

๐Ÿ“„

Semantic Wikipedia Search with Transformers and DistilBERT

Brand new to neural search? See a simple text-search example to understand how Jina works

๐Ÿ“„

Add Incremental Indexing to Wikipedia Search

Index more effectively by adding incremental indexing to your Wikipedia search

๐Ÿ“„

Build a NLP Semantic Search System with Transformers

Upgrade from plain search to sentence search and practice your Flows and Pods by searching sentences from Wikipedia

๐Ÿ“„

Search Lyrics with Transformers and PyTorch

Get a better understanding of chunks by searching a lyrics database. Now with shiny front-end!

๐Ÿ–ผ๏ธ

Google's Big Transfer Model in (Pokรฉ-)Production

Use SOTA visual representation for searching Pokรฉmon!

๐ŸŽง

Search YouTube audio data with Vggish

A demo of neural search for audio data based Vggish model.

๐ŸŽž๏ธ

Search Tumblr GIFs with KerasEncoder

Use prefetching and sharding to improve the performance of your index and query flow when searching animated GIFs.

Please check our examples repo for advanced and community-submitted examples.

Want to read more? Check our Founder Han Xiao's blog and our official blog.

Documentation

Apart from the learning resources we provided above, We highly recommended you go through our documentation to master Jina.

Our docs are built on every push, merge, and release of Jina's master branch. Documentation for older versions is archived here.

Are you a "Doc"-star? Join us! We welcome all kinds of improvements on the documentation.

Contributing

We welcome all kinds of contributions from the open-source community, individuals and partners. We owe our success to your active involvement.

Contributors โœจ

All Contributors

Community

  • Code of conduct - play nicely with the Jina community
  • Slack workspace - join #general on our Slack to meet the team and ask questions
  • YouTube channel - subscribe to the latest video tutorials, release demos, webinars and presentations.
  • LinkedIn - get to know Jina AI as a company and find job opportunities
  • Twitter Follow - follow and interact with us using hashtag #JinaSearch
  • Company - know more about our company and how we are fully committed to open-source.

Open Governance

As part of our open governance model, we host Jina's Engineering All Hands in public. This Zoom meeting recurs monthly on the second Tuesday of each month, at 14:00-15:30 (CET). Everyone can join in via the following calendar invite.

The meeting will also be live-streamed and later published to our YouTube channel.

Join Us

Jina is an open-source project. We are hiring full-stack developers, evangelists, and PMs to build the next neural search ecosystem in open source.

License

Copyright (c) 2020-2021 Jina AI Limited. All rights reserved.

Jina is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.

Comments
  • Text-by-Audio

    Text-by-Audio

    Hi,

    I want to retrieve text by searching for an audio using AudioClip model.

    First, I created indexing of text (car-horn, coughing, alarm-clock, thunderstorm etc) and using AudioClipTextEncoder for embedding.

    After that I am searching for an audio where i am using AudioClipEncoder for embedding.

    For both text and audio indexing simple-indexer is used.

    While searching for an audio i created chunks using segmenter and using MyRanker for ranking scores where i modified some script.

    The output is: input_audio: AudioCLIP/demo/audio/thunder_3-144891-B-19.wav, len-of-chunks:1 +- id: e5eceb26737311ec83fd2fdc0fc83b8e score: 0.931110, cat input_audio: AudioCLIP/demo/audio/coughing_1-58792-A-24.wav, len-of-chunks:1 +- id: e5ecf968737311ec83fd2fdc0fc83b8e score: 0.923869, cat input_audio: AudioCLIP/demo/audio/cat_3-95694-A-5.wav, len-of-chunks:1 +- id: e5ed00de737311ec83fd2fdc0fc83b8e score: 0.845475, cat input_audio: AudioCLIP/demo/audio/car_horn_1-24074-A-43.wav, len-of-chunks:1 +- id: e5ed0778737311ec83fd2fdc0fc83b8e score: 0.928283, cat input_audio: AudioCLIP/demo/audio/alarm_clock_3-120526-B-37.wav, len-of-chunks:1 +- id: e5ed0dcc737311ec83fd2fdc0fc83b8e score: 0.919222, alarm_clock

    As above for 5 inputs only 2 outputs are correct. Please let me know why is that so ? Since AudioClipEncoder is giving correct outputs for all audios.

    opened by snaaz21 55
  • [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

    [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ]

    Describe your proposal/problem

    Dear Jina Team,

    Recently, I tried nlp-simple example and replaced TransformerTorchEncoder with TextPaddlehubEncoder. However, I got the following error.

    Traceback (most recent call last):
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 73, in _load_executor
        self._executor = BaseExecutor.load_config(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 531, in load_config
        return JAML.load(tag_yml, substitute=False)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 89, in load
        r = yaml.load(stream, Loader=JinaLoader)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/__init__.py", line 114, in load
        return loader.get_single_data()
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 51, in get_single_data
        return self.construct_document(node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 55, in construct_document
        data = self.construct_object(node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/yaml/constructor.py", line 100, in construct_object
        data = constructor(self, node)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/__init__.py", line 422, in _from_yaml
        return get_parser(cls, version=data.get('version', None)).parse(cls, data)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/jaml/parsers/executor/legacy.py", line 130, in parse
        obj = cls(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 82, in __call__
        getattr(obj, '_post_init_wrapper', lambda *x: None)(m, r)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/executors/__init__.py", line 174, in _post_init_wrapper
        self.post_init()
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/jina/hub/encoders/nlp/TextPaddlehubEncoder/__init__.py", line 38, in post_init
        self.model = hub.Module(name=self.model_name)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 171, in __new__
        module = cls.init_with_name(
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 263, in init_with_name
        user_module_cls = manager.install(name=name, version=version, source=source, update=update, branch=branch)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 188, in install
        return self._install_from_name(name, version)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 263, in _install_from_name
        return self._install_from_url(item['url'])
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 256, in _install_from_url
        return self._install_from_archive(file)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 361, in _install_from_archive
        return self._install_from_directory(directory)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/manager.py", line 345, in _install_from_directory
        hub_module_cls = HubModule.load(self._get_normalized_path(module_info.name))
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddlehub/module/module.py", line 219, in load
        paddle.set_device(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/device.py", line 166, in set_device
        framework._set_expected_place(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 317, in _set_expected_place
        _set_dygraph_tracer_expected_place(place)
      File "/home/work/renyuanhang/my_anaconda3/envs/jina_rocket2/lib/python3.8/site-packages/paddle/fluid/framework.py", line 311, in _set_dygraph_tracer_expected_place
        _dygraph_tracer_._expected_place = place
    OSError: (External)  Cuda error(3), initialization error.
      [Advise: The API call failed because the CUDA driver and runtime could not be initialized. ] (at /paddle/paddle/fluid/platform/gpu_info.cc:229)
    

    It seems to be related with https://github.com/PaddlePaddle/Paddle/issues/25185.

    However, I have no idea how to fix this. Could you help me to figure it out? Thank you very much!


    Environment

    paddlehub==2.0.0 paddlenlp==2.0.1 paddlepaddle-gpu==2.0.2.post90 jina==1.3.0

    Screenshots

    image

    area/core type/bug 
    opened by ryh95 47
  • feat(instrumentation): add OpenTelemetry tracing and metrics with basic configurations

    feat(instrumentation): add OpenTelemetry tracing and metrics with basic configurations

    Goals:

    • resolves #5155
    • [x] Integrate OpenTelemetry API and SDK.
    • ~[ ] Provide environment variable configurations to enable tracking when required. Use console exporter for now.~
    • [x] Trace gRPC requests within the Flow.
    • [x] Add helpers for creating traces on request methods with default span attributes.
    • ~[ ] Convert send_health_check_sync or is_ready method to async to prevent the grpc aio interceptor from throwing and capturing an exception.~
    • [x] Extract tracing context from the server and make it available for the Executor methods in the kwargs list or arguments.
    • [ ] check and update documentation. See guide and ask the team.

    Sample Usage

    Flow

    jtype: Flow
    version: '1'
    with:
      protocol: grpc
      port: 54321
      tracing: true
      traces_exporter_host: '0.0.0.0'
      traces_exporter_port: 4317
    executors:
      - uses: executor1/config.yml
        name: toyExecutor
      - uses: executor2/config.yml
        name: toyExecutor2
    

    Executor

    import functools
    
    from jina import DocumentArray, Executor, requests
    from opentelemetry.context.context import Context
    from opentelemetry.semconv.trace import SpanAttributes
    from opentelemetry.trace import Status, StatusCode
    
    
    class MyExecutor(Executor):
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
    
        @requests
        def foo(self, docs: DocumentArray, tracing_context: Context, **kwargs):
            with self.tracer.start_span("foo", context=tracing_context) as span:
                try:
                    span.set_attribute("len_added_docs", len(docs))
                    span.set_attribute(SpanAttributes.RPC_METHOD, functools.__name__)
    
                    docs[0].text = 'hello, world!'
                    docs[1].text = 'goodbye, world!'
                except Exception as ex:
                    span.set_status(Status(StatusCode.ERROR))
                    span.record_exception(ex)
                finally:
                    span.set_status(Status(StatusCode.OK))
    

    Client

    from jina import Client, DocumentArray
    import time
    
    if __name__ == '__main__':
        c = Client(
            host='grpc://0.0.0.0:54321',
            tracing=True,
            traces_exporter_host='0.0.0.0',
            traces_exporter_port=4317,
        )
    
        da = c.post('/', DocumentArray.empty(4))
        print(da.texts)
    
        time.sleep(3)
    
    

    Collecting Data

    Please check the docker-compose.yml and otel-collector-config.yml under the folder tests/integration/instrumentation for running the OpenTelemetry collector and Jaeger UI locally.

    size/L area/core size/S area/testing size/XL area/docs component/resource area/setup component/client area/cli 
    opened by girishc13 45
  • No documents are found with BinaryPbIndexer in 1.1.2

    No documents are found with BinaryPbIndexer in 1.1.2

    Versions 1.0.11: everything works as expected 1.1.2: BinaryPbIndexer doesn't find any documents

    Detailed description He Jina team, when I upgrade to 1.1.2 and reindex my dataset, the BinaryPbIndexer no longer works. It doesn't raise any errors, but I just can't find any documents. Just as a remark, I use custom ids which always consist of 33 characters. Where there any changes for the BinaryPbIndexer, the KVIndexDriver or the KVSearchDriver?

    area/core priority/critical type/bug status/done 
    opened by janandreschweiger 39
  • Permanently delete documents from storage

    Permanently delete documents from storage

    Hey Jina team!

    As described here documents persist the deletion and are only marked as inaccessible.

    My question: Is there a possibility to delete documents from storage without reindexing everything again? Deletion performance isn't an issue for us, but reindexing everything is very inconvenient in our use case.

    Thanks! I really appreciate your kind support.

    Sub-tasks

    • [x] Delete from NpIndexer. @ #2046
    • [x] Delete from BinaryPbIndexer. @ https://github.com/jina-ai/jina/pull/2102
    area/core priority/important-soon type/feature-request status/done 
    opened by ace-kay-law-neo 39
  • File seems to be closed when trying to index

    File seems to be closed when trying to index

    Yes @JoanFM, using the prefetch argument in the flow-config solved the original problem. I tried to pass the batch_size and the request_size like you, but I still get the same error. I have some additional logs, which might indicate what causes the problem. As I said, the first few batches are indexed successfully, but at some point I get this error:

            [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
            [email protected][I]:#sent: 3 #recv: 4 sent_size: 13.8 MB recv_size: 1.2 MB
          [email protected][I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธโš
            [email protected][I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธโš
       [email protected][S]:artifacts of this executor (wrap_idx) is persisted to /var/huforce/fs/workspace/wrap_idx-0/wrap_idx.bin
    [email protected][S]:artifacts of this executor (chunk_idx) is persisted to /var/huforce/fs/workspace/chunk_idx-0/chunk_idx.bin
          [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
          [email protected][I]:#sent: 3 #recv: 4 sent_size: 1.9 MB recv_size: 15.6 MB
            [email protected][I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธdoc_idx/ZEDRuntimeโ–ธโš
    DEBUG | Message: Successfully indexed 4 documents!
            [email protected][S]:artifacts of this executor (doc_idx_file) is persisted to /var/huforce/fs/workspace/doc_idx_file-0/doc_idx_file.bin
            [email protected][I]:#sent: 3 #recv: 4 sent_size: 6.3 KB recv_size: 2.2 MB
           | ๐Ÿ“ƒ     16 โฑ๏ธ 262.9s ๐ŸŽ 0.1/s      4   requests        [email protected][I]:send: 5 recv: 4 pending: 1
            [email protected][I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธโš
    DEBUG | Message: File-Info: research_admission.txt
    DEBUG | Message: File-Info: cloudcomputing.pptx
    DEBUG | Message: File-Info: WS_2012_White_Paper_Hyper-V.pdf
    DEBUG | Message: File-Info: writing-sql-queries.docx
            [email protected][I]:no update since 2021-04-10 07:38:17, will not save. If you really want to save it, call "touch()" before "save()" to force saving
            [email protected][I]:#sent: 5 #recv: 6 sent_size: 1.2 MB recv_size: 5.2 KB
            [email protected][I]:#sent: 4 #recv: 5 sent_size: 15.6 MB recv_size: 1.2 MB
          [email protected][I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโ–ธโš
            [email protected][I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธโš
          [email protected][I]:#sent: 4 #recv: 5 sent_size: 2.2 MB recv_size: 16.1 MB
    DEBUG | Message: Encoding 552 paragraphs
          [email protected][E]:ValueError('write to closed file')
     add "--quiet-error" to suppress the exception details
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 198, in _msg_callback
        self._zmqlet.send_message(self._callback(msg))
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 184, in _callback
        self._pre_hook(msg)._handle(msg)._post_hook(msg)
      File "/usr/local/lib/python3.8/site-packages/jina/peapods/runtimes/zmq/zed.py", line 177, in _handle
        self._executor(self.request_type)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/__init__.py", line 595, in __call__
        d()
      File "/usr/local/lib/python3.8/site-packages/jina/drivers/__init__.py", line 381, in __call__
        self._apply_all(documents, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/jina/drivers/index.py", line 58, in _apply_all
        self.exec_fn(keys, values)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/decorators.py", line 44, in arg_wrapper
        f = func(self, *args, **kwargs)
      File "/usr/local/lib/python3.8/site-packages/jina/executors/indexers/keyvalue.py", line 178, in add
        self.write_handler.header.write(
    ValueError: write to closed file
            [email protected][I]:recv IndexRequest  from gatewayโ–ธcrafter/ZEDRuntimeโ–ธencoder/ZEDRuntimeโ–ธchunk_idx/ZEDRuntimeโœ–โ–ธdoc_idx/ZEDRuntimeโ–ธโš
    dev_ai            | DEBUG | Message: Successfully indexed 4 documents!
    

    What's also strange is, that the failed documents for the chunk-idx are passed on to the doc-idx, as you can see in the logs.

    My flow for indexing:

    !AsyncFlow
    with:
      logserver: false
      show-exc-info: true
      prefetch: 2
      # request-size: 2
      # batch_size: 2
    pods:
      crafter:
        uses: pods/craft.yml
      encoder:
        uses: pods/encode.yml
        timeout_ready: 600000
      chunk_idx:
        uses: pods/chunk.yml
      doc_idx:
        uses: pods/doc.yml
    

    Originally posted by @nero-nazok in https://github.com/jina-ai/jina/issues/2298#issuecomment-817084516

    area/core type/bug 
    opened by JoanFM 32
  • DOCQA..

    DOCQA..

    I am trying to parse a pdf & apply the DOCQA model to it.Roberta as well as BERT..!! But I am not getting right answer for it.. whereas independently it's works accurately... !!

    This is my flow.yml

    jtype: Flow
    version: '1'
    with:
      workspace: $JINA_WORKSPACE
      port_expose: $JINA_PORT
    executors:
      - name: transformer
        uses: 'jinahub://TransformerTorchEncoder/v0.1'
        uses_with:
          device: 'cuda'
      - name: indexer
        uses: 'jinahub://SimpleIndexer/old'
    

    This is my query.yml

    jtype: Flow
    version: '1'
    with:
      workspace: $JINA_WORKSPACE
      port_expose: $JINA_PORT
    executors:
      - name: transformer
        uses: 'jinahub://TransformerTorchEncoder/v0.1'
        uses_with:
          device: 'cuda'
      - name: indexer
        uses: 'jinahub://SimpleIndexer/old'
      - name: generator
        uses: Generator
        py_modules: "flows/generator_roberta.py"
    
    opened by jyotikhetan 29
  • feat: allow multiple port and protocols for gateway

    feat: allow multiple port and protocols for gateway

    This PR adds support for multiple ports and protocols for the gateway runtime.

    • [x] Gateway parser can receive list or single value
    • [x] Flow __init__ and config_gateway can receive multiple or single value. Flow.port and Flow.protocol return single or multiple values
    • [x] GatewayRuntime expects list of values
    • [x] BaseGateway can receive runtime args
    • [ ] (optional) move injected dependencies to runtime_args, TBD in https://github.com/jina-ai/jina/issues/5396
    • [x] implemented Gateway classes uses the port and protocol from runtime args
    • [x] support in JAML
    • [x] k8s gateway service
    • [x] docker compose gateway service
    • [x] health checks
    • NetworkChecker
    • docker and k8s checks
    • Flow pod checks
    • [ ] ~~apply the same principle for port_monitoring~~ TBD in a separate PR: https://github.com/jina-ai/jina/issues/5352
    • [x] fix gateway output: https://github.com/jina-ai/jina/issues/5345
    • [ ] pass CI
    • [x] Add tests follow up: https://github.com/jina-ai/jina/issues/5393 closes: https://github.com/jina-ai/jina/issues/5345, https://github.com/jina-ai/jina/issues/5343
    size/M size/L area/core area/helper area/testing size/XL area/docs component/resource area/cli component/jaml 
    opened by alaeddine-13 28
  • Gateway error: received an empty stream from the client

    Gateway error: received an empty stream from the client

    Description

    I encountered an error in flow.post method while trying to run python app.py -t index in multires-lyrics-search after cloning this repository: examples. The error message was: [email protected][E]:receive an empty stream from the client! please check your client's inputs, you can use "Client.check_input(inputs)".


    Environment

    • jina 2.0.18
    • jina-proto 0.0.85
    • jina-vcs-tag (unset)
    • libzmq 4.3.4
    • pyzmq 1.21.5
    • protobuf 3.20.0
    • proto-backend cpp
    • grpcio 1.44.0
    • pyyaml 6.0
    • python 3.7.13
    • platform Linux
    • platform-release 5.10.105-1-MANJARO
    • platform-version #1 SMP PREEMPT Fri Mar 11 14:12:33 UTC 2022
    • architecture x86_64
    • processor
    • uid 5025763201072
    • session-id 7d472a12-b722-11ec-a806-049226d49030
    • uptime 2022-04-08T15:58:49.863145
    • ci-vendor (unset)
    • JINA_ARRAY_QUANT (unset)
    • JINA_CONTROL_PORT (unset)
    • JINA_DEFAULT_HOST (unset)
    • JINA_DISABLE_UVLOOP (unset)
    • JINA_FULL_CLI (unset)
    • JINA_HUBBLE_REGISTRY (unset)
    • JINA_HUB_CACHE_DIR (unset)
    • JINA_HUB_ROOT (unset)
    • JINA_LOG_CONFIG (unset)
    • JINA_LOG_ID (unset)
    • JINA_LOG_LEVEL (unset)
    • JINA_LOG_NO_COLOR (unset)
    • JINA_LOG_WORKSPACE (unset)
    • JINA_OPTIMIZER_TRIAL_WORKSPACE(unset)
    • JINA_POD_NAME (unset)
    • JINA_RANDOM_PORT_MAX (unset)
    • JINA_RANDOM_PORT_MIN (unset)
    • JINA_VCS_VERSION (unset)
    • JINA_MP_START_METHOD (unset)
    opened by Jackal1586 28
  • feat: add grpc health checking

    feat: add grpc health checking

    This PR aims to change the way health check is done in the core for Executors but also for any runtime using grpc. We aim to use the standard grpc https://github.com/grpc/grpc/blob/master/doc/health-checking.md as suggested by grpc, and which is something that lots of Load Balancers, etc ... will use to make sure grpc-based microservices are healthy.

    This also has the benefit that ControlRequest can be removed from the code.

    area/core area/helper area/testing area/network size/XL area/docs component/resource area/setup component/client component/proto area/entrypoint component/type 
    opened by JoanFM 27
  • feat: remove head for non sharded deployments

    feat: remove head for non sharded deployments

    Remove heads for non sharded Deployments

    Description

    This PR removes the head for non sharded deployments. This means if shards are set to 1 for a Deployment, there will be no head created. In these cases the Gateway communicates directly with the workers. Also reducing requests in case of multiple needs is moved to the Gateway now as we cant rely on the fact to due it at the head anymore.

    This is a breaking change :boom: It breaks uses_before/uses_after for non sharded Deploments. It also changes the way we interact with external Executors. Additionally rolling_update and scale are removed from the Flow.

    Before we do a release we should do the follow up issues around (external) Deployments as first class citizens.

    • [x] Remove RollingUpdate/Scale feature from Flow/Deployment/Pod/JinaD
    • [x] Set all worker Pods in the Gateway (Outside K8s)
    • [x] Make the gateway send to worker Pods
    • [x] Dont create heads for Deployments with no shards
    • [x] Implement reducing after multiple needs in the Gateway
    • [x] Adapt tests
    • [x] Change docs

    Closes #4503 (issue)

    area/core area/helper area/testing size/XL area/docs area/cli area/daemon 
    opened by jacobowitz 27
  • Change documentation for `CONTEXT` environment variables

    Change documentation for `CONTEXT` environment variables

    Describe your proposal/problem

    The docs don't specify how to use context variables in a flow yaml.

    It should be made clear that when defining a flow using the YAML specification VALUE_A & VALUE_B should appear in the env key.


    Flow.yml

    jtype: Flow
    executors:
      - name: executor1
        uses: executor1/config.yml
        env:
          VALUE_A: 123
          VALUE_B: hello
        uses_with:
          var_a: ${{ CONTEXT.VALUE_A }}
          var_b: ${{ CONTEXT.VALUE_B }}
    
    area/docs 
    opened by npitsillos 0
  • Refactor: use K8sDeploymentConfig inside Deployment to generate kubernetes yamls

    Refactor: use K8sDeploymentConfig inside Deployment to generate kubernetes yamls

    After Deployment is exposed to serve Executors, we want to use it in order to generate kubernetes yaml as well. This means, the Deployment should use K8sDeploymentConfig to implement to_kubernetes_yaml.

    Potentially, we also want to refactor Flow.to_kubernetes_yaml to use the new method and maybe the same for Executor (or deprecate Executor.to_kubernetes_yaml)

    opened by alaeddine-13 0
  • fix(hub-integration): apply changes needed for Executor arm64 support

    fix(hub-integration): apply changes needed for Executor arm64 support

    Goals:

    • jina-hubble-sdk recently provided --platform parameter for Executor push and --prefer-platform for Executor pull
    • Update related tests to new version of fetch_meta provided by the SDK
    • Automated updates of command-line arguments
    • [ ] check and update documentation. See guide and ask the team.
    area/core size/S area/testing area/docs size/XS area/cli 
    opened by nomagick 2
  • feat: run warmup on Runtimes and Executor

    feat: run warmup on Runtimes and Executor

    Goals:

    • resolves #ISSUE-NUMBER
    • small cleanup
    • warmup header runtime
    • use asyncio.sleep
    • [ ] check and update documentation. See guide and ask the team.
    size/M area/core area/testing area/cli 
    opened by girishc13 1
Releases(v3.13.2)
  • v3.13.2(Dec 30, 2022)

    Release Note (3.13.2)

    This release contains 1 bug fix.

    ๐Ÿž Bug Fixes

    Respect timeout_ready when generating startup probe (#5560)

    As Kubernetes Startup Probes were added to all deployments in release v3.13.0, we added default values for all probe configurations. However, if those default configurations were not enough to wait for an Executor that takes time to load and become ready, the Executor deployment would become subject to the configured restart policy. Therefore, Executors that are slow to load would keep restarting forever.

    In this patch, this behavior is fixed by making sure that Startup Probe configurations respect the timeout_ready argument of Executors. Startup Probes are configured like so:

    • periodSeconds always set to 5 seconds
    • timeoutSeconds always set to 10 seconds
    • failureThreshold is the number of attempts maybe by kubernetes to check if the pod is ready. It varies according to timeout_ready. The formula used is failureThreshold = timeout_ready / 5000 (as timeout_ready is in microseconds and periodSeconds is 5 seconds) and in all cases, it will be at least 3. If timeout_ready is -1 (in Jina it means waiting forever for the Executor to become ready), since waiting forever is not supported in Kubernetes, the value for failureThreshold will be 120 attempts.

    ๐Ÿค˜ Contributors

    We would like to thank all contributors to this release:

    Source code(tar.gz)
    Source code(zip)
  • v3.13.1(Dec 21, 2022)

    Release Note (3.13.1)

    Release time: 2022-12-21 12:58:15

    This release contains 3 bug fixes and 1 documentation improvement.

    ๐Ÿž Bug Fixes

    Support Gateway with multiple protocols for Kubernetes export (#5532)

    You can now export Flows with multiple protocols to Kubernetes. Previously this would cause an error.

    flow = Flow().config_gateway(protocol=['http', 'grpc'])
    flow.to_kubernetes_yaml('k8s_flow_folder')
    

    Fix Python 3.11 support (#5529)

    It was previously impossible to install Jina with Python 3.11 due to a grpcio dependency problem. grpcio added support for Python 3.11 only with version 1.49.0, causing potential problems when used by Jina and other projects.

    In this release grpcio>=1.49.0 is installed alongside Jina when using Python 3.11. However, be aware of potential problems related to grpc hanging.

    Unary RPC from Client respects results_in_order (#5513)

    In prior releases, calling the post method of a client with grpc and using stream=False did not respect the results_in_order parameter and results were always returned in order:

    # this wrongly returns results in order
    c = Client(protocol='grpc')
    c.post(on='/', inputs=DocumentArray.empty(10000), stream=False, results_in_order=False)  
    

    Also this implied that using the Client with asyncio=True and stream=False in the post call would return results in the order that they were returned by the Flow, rather than respecting the input order:

    # this wrongly returns results in order
    c = Client(protocol='grpc', asyncio=True)
    
    async for resp in c.post(on='/', inputs=DocumentArray.empty(10000), stream=False, results_in_order=False)
        print(resp)
    

    This release fixes the ordering bug.

    ๐Ÿ“— Documentation Improvements

    • Document inheritance of arguments from Flow API to Executors and Gateway (#5535)

    ๐Ÿค˜ Contributors

    We would like to thank all contributors to this release:

    Source code(tar.gz)
    Source code(zip)
  • v3.13.0(Dec 15, 2022)

    Release Note (3.13.0)

    Release time: 2022-12-15 15:33:43

    This release contains 14 new features, 9 bug fixes and 7 documentation improvements.

    This release introduces major features like Custom Gateways, Dynamic Batching for Executors, development support with auto-reloading, support for the new namespaced Executor scheme jinaai, improvements for our gRPC transport layer, and more.

    ๐Ÿ†• Features

    Custom Gateways (#5153, #5189, #5342, #5457, #5465, #5472 and #5477)

    Jina Gateways are now customizable in the sense that you can implement them in much the same way as an Executor. With this feature, Jina gives power to the user to implement any server, protocol or interface at the Gateway level. There's no more need to build an extra service that uses the Flow.

    For instance, you can define a Jina Gateway that communicates with the rest of Flow Executors like so:

    from docarray import Document, DocumentArray
    from jina.serve.runtimes.gateway.http.fastapi import FastAPIBaseGateway
    
    
    class MyGateway(FastAPIBaseGateway):
        @property
        def app(self):
            from fastapi import FastAPI
    
            app = FastAPI(title='Custom FastAPI Gateway')
    
            @app.get(path='/service')
            async def my_service(input: str):
                # convert input request to Documents
                docs = DocumentArray([Document(text=input)])
    
                # send Documents to Executors using GatewayStreamer
                result = None
                async for response_docs in self.streamer.stream_docs(
                    docs=docs,
                    exec_endpoint='/',
                ):
                    # convert response docs to server response and return it
                    result = response_docs[0].text
    
                return {'result': result}
    
            return app
    

    Then you can use it in your Flow in the following way:

    flow = Flow().config_gateway(uses=MyGateway, port=12345, protocol='http')
    

    A Custom Gateway can be used as a Python class, YAML configuration or Docker image.

    Adding support for Custom Gateways required exposing the Gateway API and supporting multiple ports and protocols (mentioned in a prior release). You can customize it by subclassing the FastAPIBaseGateway class (for simple implementation) or base Gateway for more complex use cases.

    Working on this feature also involved exposing and improving the GatewayStreamer API as a way to communicate with Executors within the Gateway.

    Find more information in the Custom Gateway page.

    Dynamic batching (#5410)

    This release adds Dynamic batching capabilities to Executors.

    Dynamic batching allows requests to be accumulated and batched together before being sent to an Executor. The batch is created dynamically depending on the configuration for each endpoint.

    This feature is especially relevant for inference tasks where model inference is more optimized when batched to efficiently use GPU resources.

    You can configure Dynamic batching using either a decorator or the uses_dynamic_batching parameter. The following example shows how to enable Dynamic batching on an Executor that performs model inference:

    from jina import Executor, requests, dynamic_batching, Flow, DocumentArray, Document
    import numpy as np
    import torch
    
    
    class MyExecutor(Executor):
        def __init__(self, **kwargs):
            super().__init__(**kwargs)
    
            # initialize model
            self.model = torch.nn.Linear(in_features=128, out_features=128)
    
        @requests(on='/bar')
        @dynamic_batching(preferred_batch_size=10, timeout=200)
        def embed(self, docs: DocumentArray, **kwargs):
            docs.embeddings = self.model(torch.Tensor(docs.tensors))
    
    
    flow = Flow().add(uses=MyExecutor)
    

    With Dynamic Batching enabled, the Executor above will efficiently use GPU resources to perform inference by batching Documents together.

    Read more about the feature in the Dynamic Batching documentation page.

    Install requirements of local Executors (#5508)

    Prior to this release, the install_requirements parameter of Executors only installed Executor requirements for Hub Executors. Now, local Executors with a requirements.txt file will also have their requirements installed before starting Flows.

    Support jinaai Executor scheme to enable namespaced Hub Executors (#5462, #5468 and #5515)

    As Jina AI Cloud introduced namespaces to Executor resources, we made changes to support the new jinaai Executor scheme. This PR adds support for the new scheme.

    This means that namespaced Executors can now be used with the jinaai scheme in the following way:

    from jina import Flow
    
    flow = Flow().add(uses='jinaai://jina-ai/DummyHubExecutor')
    

    This scheme is also supported in Kubernetes and other APIs:

    from jina import Flow
    
    flow = Flow().add(uses='jinaai+docker://jina-ai/DummyHubExecutor')
    flow.to_kubernetes_yaml('output_path', k8s_namespace='my-namespace')
    

    The support of the new scheme means the minimum supported version of jina-hubble-sdk has been increased to 0.26.10.

    Add auto-reloading to Flow and Executor on file changes (#5461, #5488 and #5514)

    A new argument reload has been added to the Flow and Executor APIs, which automatically reloads running Flows and Executors when changes are made to Executor source code or YAML configurations of Flows and Executors.

    Although this feature is only meant for development, it aims to help developers iterate fast and automatically update Flows with changes they make live during development.

    Find out more about this feature in these two sections:

    Expand Executor serve parameters (#5494)

    The method Executor.serve can receive more parameters, similar to what the Flow API expects. With new parameters to control serving and deployment configurations of the Executor, this method empowers the Executor to be convenient for single service tasks.

    This means you can not only build advanced microservices-based pipelines and applications, but also build individual services with all Jina features: shards/replicas, dynamic batching, auto-reload, etc.

    Read more about the method in the Python API documentation.

    Add gRPC trailing metadata when logging gRPC error (#5512)

    When logging gRPC errors, context trailing metadata is now shown. This helps identify underlying network issues rather than the error codes that mask multiple network errors into a single gRPC status code.

    For instance, the new log message looks like the following:

    DEBUG  gateway@ 1 GRPC call to deployment executor0 failed                      
           with error <AioRpcError of RPC that terminated with:                     
                   status = StatusCode.UNAVAILABLE                                  
                   ...                                                                   
           trailing_metadata=Metadata((('content-length', '0'),                     
           ('l5d-proxy-error', 'HTTP Balancer service in                            
           fail-fast'), ('l5d-proxy-connection', 'close'),                          
           ('date', 'Tue, 13 Dec 2022 10:20:15 GMT'))), for                         
           retry attempt 2/3. Trying next replica, if available. 
    

    The trailing_metadata returned by load balancers will help to identify the root cause more accurately.

    Implement unary_unary stub for Gateway Runtime (#5507)

    This release adds the gRPC unary_unary stub for Gateway Runtime as a new communication stub with Executors. Since the gRPC performance best practices for Python page suggests that unary stream implementation might be faster for Python, we added this communication method.

    However, this is not enabled by default. The streaming RPC method will still be used unless you set the stream option to False in the Client.post() method. The feature is only effective when the gRPC protocol is used.

    Read more about the feature in the documentation: https://docs.jina.ai/concepts/client/send-receive-data/#use-unary-or-streaming-grpc

    Add Startup Probe and replace Readiness Probe with Liveness Probe (#5407)

    Before this release, when exporting Jina Flows to Kubernetes YAML configurations, Kubernetes Readiness Probes used to be added for the Gateway pod and each Executor pod. In this release we have added a Startup Probe and replaced Readiness Probe with Liveness Probe.

    Both probes use the jina ping command to check that pods are healthy.

    New Jina perf Docker images (#5498)

    We added a slightly larger Docker image with suffix perf which includes a set of tools useful for performance tuning and debugging.

    The new image is available in Jina AI's Docker hub.

    New Jina Docker image for Python 3.10, and use Python 3.8 for default Jina image (#5490)

    Besides adding Docker images aimed for performance optimization, we added an image with a newer Python version: 3.10. This is available in Jina AI's Docker hub, for example jinaai/jina:master-py310-standard.

    We also made Python 3.8 our minimum supported Python version by default, and it will be used for default Docker images.

    Minimize output of jina ping command (#5476)

    jina ping commands are now less verbose and will print less irrelevant output. However, important information like latency for each round, average latency, number of successful requests and ping result will still show up.

    Add Kubernetes preStop hook to the containers (#5445)

    A preStop hook has been added to Executors and the Gateway to allow a grace period. This allows more time to complete in-flight requests and finish the server's graceful shutdown.

    Generate random ports for multiple protocols (#5455)

    If you use multiple protocols for a Gateway, you no longer need to specify a port for each one. Whether it's Python or YAML, you just need to specify the protocols you want to support and Jina will generate random ports for you.

    Python API:

    from jina import Flow
    
    flow = Flow().config_gateway(protocol=['grpc', 'http', 'websocket'])
    with flow:
        flow.block()
    

    YAML:

    jtype: Flow
    gateway:
      protocol:
        - 'grpc'
        - 'http'
        - 'websocket'
    

    Result: flow

    ๐Ÿž Bug Fixes

    List-like args passed as string (#5464)

    We fixed the format expected for port, host and port_monitoring to feel more Pythonic. Basically, if you use replicas, you no longer have to provide comma-separated ports as a string value. Instead, you can simply pass a list of values, no need to put all in a string anymore!

    For instance, suppose we have two external replicas of an Executor that we want to join in our Flow (the first is hosted on localhost:12345 and the second on 91.198.174.192:12346). We can add them like this:

    from jina import Flow
    
    replica_hosts, replica_ports = ['localhost', '91.198.174.192'], [
        '12345',
        '12346',
    ]  # instead of 'localhost,91.198.174.192', '12345,12346'
    Flow().add(host=replica_hosts, port=replica_ports, external=True)
    

    Or:

    Flow().add(host=['localhost:12345', '91.198.174.192:12346'], external=True)
    

    Note that this is not a breaking change, and the old syntax (comma-separated values: Flow().add(host='localhost:12345,91.198.174.192:12346', external=True)) is still supported for backwards compatibility.

    Restore port to overload type hint and JSON schema (#5501)

    When we made port and protocol arguments of the Gateway support multiple values, a bug was introduced where port did not appear in Jina's JSON schema as well as the Flow API overload for method signatures.

    Although the arguments are functional in both the Python API and YAML, this suppressed auto-completion and developer support for these parameters. This release restores the port parameter in both the Flow method overloads and JSON schema.

    Do not force insecure to True in open telemetry integration (#5483)

    In Jina's instrumentation, communication to open telemetry exporters used to be forced to insecure mode. Luckily, our community member @big-thousand picked this up and submitted a fix. The communication is no longer forced to the insecure mode.

    Kudos to @big-thousand for his contribution!

    Fix problem when using floating Executor in HTTP (#5493)

    We found a bug when using Floating Executors in HTTP, where the floating Executor is connected to the Gateway (in the Flow topology). In this case, the Executor would not receive input Documents properly. This release fixes the mentioned bug.

    Add egg info post install command for egg info setup mode (#5491)

    This release adds support for the egg info setup mode in Python. This means post-installation commands are now properly executed in environments that rely on Python's new setup mode.

    This bug resulted in several issues especially for environments that depend on these post-installation commands. For instance, some Environment Variables that are needed for Jina to work on macOS and for CLI auto-complete.

    Do not apply limits when gpus='all' in Kubernetes (#5485)

    If Executor parameter gpus is set to "all", no limits will be applied on the pod in Kubernetes.

    Fix Windows signal handling (#5484)

    This release improves signal handling on Windows, specifically when cancelling a Flow with an OS signal.

    Cap opentelemetry-instrumentation-aiohttp-client (#5452)

    This release caps the version for opentelemetry-instrumentation-aiohttp-client which is incompatible with opentelemetry-semantic-conventions.

    Raise exceptions from path importer (#5447)

    Previously, errors were hidden when they came from a Python module imported to load an Executor. Actually the module was not considered to be a Python module, which produced other misleading errors. In this release, actual errors during imports will be raised and no longer hidden.

    ๐Ÿ“— Documentation Improvements

    • Add gRPC requirements for Apple Silicon (M1 Chip) to fix failing installation of Jina (#5511)
    • Add redirects from '/fundamentals' to '/concepts' (#5504)
    • Update JCloud documentation to the jcloud v0.1.0 (#5385)
    • Restructure documentation under /concepts
    • Change Executor URI scheme to namespaced scheme jinaai (#5450)
    • Custom Gateway documentation (#5465)
    • Provide more accurate description for port and protocol parameters of the Gateway (#5456)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Delgermurun (@delgermurun)
    • Jie Fu (@jemmyshin)
    • Alex Cureton-Griffiths (@alexcg1)
    • big-thousand (@big-thousand)
    • IyadhKhalfallah (@IyadhKhalfallah)
    • Deepankar Mahapatro (@deepankarm)
    • samsja (@samsja)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Joan Fontanals (@JoanFM)
    • Anne Yang (@AnneYang720)
    • Han Xiao (@hanxiao)
    • Girish Chandrashekar (@girishc13)
    • Jackmin801 (@Jackmin801)
    Source code(tar.gz)
    Source code(zip)
  • v3.12.0(Nov 25, 2022)

    Release Note (3.12.0)

    Release time: 2022-11-25 12:28:29

    This release contains 8 new features, 16 bug fixes and 15 documentation improvements.

    ๐Ÿ†• Features

    Support multiple protocols at the same time in Flow Gateways (#5435 and #5378)

    Prior to this release, a Flow only exposed one server in its Gateway with one of the following protocols: HTTP, gRPC or WebSockets.

    Now, you can specify multiple protocols and for each one, a separate server is started. Each server is bound to its own port.

    For instance, you can do:

    from jina import Flow
    flow = Flow(port=[12345, 12346, 12347], protocol=['http', 'grpc', 'websocket'])
    with flow:
        flow.block()
    

    or: jina flow --uses flow.yml where flow.yml is:

    jtype: Flow
    with:
      protocol:
        - 'grpc'
        - 'http'
        - 'websocket'
      port:
        - 12345
        - 12344
        - 12343
    

    multi-protocol-flow

    The protocol and port parameters can still accept single values rather than a list. Therefore, there is no breaking change.

    Alias parameters protocols and ports are also defined:

    flow = Flow(ports=[12345, 12346, 12347], protocols=['http', 'grpc', 'websocket'])
    

    In Kubernetes, this exposes separate services for each protocol.

    Read the docs for more information.

    Add Kubernetes information to resource attributes in instrumentation (#5372)

    When deploying to Kubernetes, the Gateway and Executors expose the following Kubernetes information as resource attributes in instrumentation:

    • k8s.namespace.name
    • k8s.pod.name
    • k8s.deployment.name / k8s.statefulset.name

    Besides that, the following resource attributes are set if they are present in the environment variables of the container:

    • k8s.cluster.name (set K8S_CLUSTER_NAME environment variable)
    • k8s.node.name (set K8S_NODE_NAME environment variable)

    Add option to return requests in order using the Client (#5404)

    If you use replicated Executors, those which finish processing first return their results to the Gateway which then returns them to the client. This is useful if you want results as soon as each replicated Executor finishes processing your Documents.

    However, this may be inconvenient if you want the Documents you send to the Flow to return in order. In this release, you can retain the order of sent Documents (when using replicated Executors) by passing the results_in_order parameter in the Client.

    For instance, if your Flow looks like this:

    from jina import Flow, DocumentArray, Document
    f = Flow().add(replicas=2)
    

    You can do the following to keep results in order:

    input_da = DocumentArray([Document(text=f'{i}') for i in range(100)])
    
    with f:
        result_da = f.post('/', inputs=input_da, request_size=10,  results_in_order=True)
        assert result_da[:, 'text'] == input_da[:, 'text']
    

    Add docs_map parameter to Executor endpoints (#5366)

    Executor endpoint signatures are extended to the following:

    class MyExecutor(Executor):
        @requests
        async def foo(
            self, docs: DocumentArray, parameters: Dict, docs_matrix: Optional[List[DocumentArray]], docs_map: Optional[Dict[str, DocumentArray]]
        ) -> Union[DocumentArray, Dict, None]:
            pass
    

    Basically, the parameter docs_map has been added. It's a dictionary that maps previous Executor names to DocumentArrays. This is useful when you have an Executor that combines results from many previous Executors, and you need information about where each resulting DocumentArray comes from.

    Add Gateway API (#5342)

    Prior to this release, all Gateway configurations were specified in the Flow API. However, by principle, Flow parameters are commonly inherited by Executors and the Gateway. We already gave the Executor its own API to be customized (either using the method add() or the executors YAML section in Flow YAML).

    In this release, we have done the same for Gateway. It defines its own API in both the Python API and YAML interface. In the Python API, you can configure the Gateway using the config_gateway() method:

    flow = Flow().config_gateway(port=12345, protocol='http')
    

    And in the YAML interface, you can configure the Gateway using the gateway section:

    !Flow
    gateway:
      protocol: http
      port: 12344
    executors:
      - name: exec
    

    This is useful when you want to apply parameters just for the Gateway. If you want a parameter to be applied to all Executors, then continue to use the Flow API.

    Keep in mind that you can still provide Gateway parameters using the Flow API. This means there are no breaking changes introduced.

    Support UUID in CUDA_VISIBLE_DEVICES round-robin assignment (#5360)

    You can specify a comma-separated list of GPU UUIDs in the CUDA_VISIBLE_DEVICES to assign devices to Executor replicas in a round-robin fashion. For instance:

    CUDA_VISIBLE_DEVICES=RRGPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5,GPU-0bbbbbbb-74d2-7297-d557-12771b6a79d5,GPU-0ccccccc-74d2-7297-d557-12771b6a79d5,GPU-0ddddddd-74d2-7297-d557-12771b6a79d5
    

    Check CUDA's documentation to see the accepted formats to assign CUDA devices by UUID.

    | GPU device | Replica ID | |------------------------------------------|------------| | GPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5 | 0 | | GPU-0bbbbbbb-74d2-7297-d557-12771b6a79d5 | 1 | | GPU-0ccccccc-74d2-7297-d557-12771b6a79d5 | 2 | | GPU-0ddddddd-74d2-7297-d557-12771b6a79d5 | 3 | | GPU-0aaaaaaa-74d2-7297-d557-12771b6a79d5 | 4 |

    Thanks to our community member @mchaker for submitting this feature request!

    Capture shard failures in the head runtime (#5338)

    In case you use Executor shards, partially failed requests (those that fail on a subset of the shards) no longer raise an error.

    Instead, successful results are returned. An error is raised only when all shards fail to process Documents. Basically, the HeadRuntime's behavior is updated to fail only when all shards fail.

    Thanks to our community user @soumil1 for submitting this feature request.

    Add successful, pending and failed metrics to HeadRuntime (#5374)

    More metrics have been added to the Head Pods:

    • jina_number_of_pending_requests: number of pending requests
    • jina_successful_requests: number of successful requests
    • jina_failed_requests: number of failed requests
    • jina_received_request_bytes: the size of received requests in bytes
    • jina_sent_response_bytes: the size of sent responses in bytes

    See more in the instrumentation docs.

    Add deployment label in grpc stub metrics (#5344)

    Executor metrics used to show up aggregated at the Gateway level and users couldn't see separate metrics per Executor. With this release, we have added labels for Executors so that metrics in the Gateway can be generated per Executor or aggregated over all Executors.

    ๐Ÿž Bug Fixes

    Check whether the deployment is in Executor endpoints mapping (#5440)

    This release adds an extra check in the Gateway when sending requests to deployments: The Gateway sends requests to the deployment only if it is in the Executor endpoint mapping.

    Unblock event loop to allow health service (#5433)

    Prior to this release, sync function calls inside Executor endpoints blocked the event loop. This meant that health-checks submitted to Executors failed for long tasks (for instance, inference using a large model).

    In this release, such tasks no longer block the event loop. While concurrent requests to the same Executor wait until the sync task finishes, other runtime tasks remain functional, mainly health-checks.

    Dump environment variables to string for Kubernetes (#5430)

    Environment variables are now cast to strings before dumping them to Kubernetes YAML.

    Unpin jina-hubble-sdk version (#5412)

    This release frees (unpins) jina-hubble-sdk version. The latest jina-hubble-sdk is installed with the latest Jina.

    Bind servers to host argument instead of __default_host__ (#5405)

    This release makes servers at each Jina pod (head, Gateway, worker) bind to the host address specified by the user, instead of always binding to the __default_host__ corresponding to the OS. This lets you, depending on your network interface, restrict or expose your Flow services in your network.

    For instance, if you wish to expose all pods to the internet, except for the last Executor, you can do:

    flow = Flow(host='0.0.0.0').add().add(host='127.0.0.1')
    

    After this fix, Jina respects this syntax and binds the last Executor only to 127.0.0.1 (accessible only inside the host machine).

    Thanks to @wqh17101 for reporting this issue!

    Fix backoff_multiplier format when using max_attempts in the Client (#5403)

    This release fixes the format of backoff_multiplier parameter when injected into the gRPC request. The issue appeared when you use the max_attempts parameter in Client.

    Maintain the correct tracing operations chain (#5391)

    Tracing spans for Executors used to show up out of order. This behavior has been fixed by using the method start_as_current_span instead of start_span to maintain the tracing chain in the correct order.

    Use Async health servicer for tracing interceptors when tracing is enabled (#5392)

    When tracing is enabled, health checks in Docker and Kubernetes deployments used to fail silently until the Flow timed out. This happened because tracing interceptors expected RPC stubs to be coroutines.

    This release fixes this issue by using the async aio.HealthServicer instead of grpc_health.HealthServicer. Health checks submitted to runtimes (Gateway, head, worker) no longer fail when tracing is enabled.

    Properly update requests count in case of Exception inside the HeadRuntime (#5383)

    In case of an Exception being raised in the HeadRuntime, request counts were not updated properly (pending requests should have been decremented and failed requests should have been incremented). This is fixed in this release and the Exception is caught to update request counts.

    Fix endpoint binding when inheriting Executors (#5380)

    When an Executor is inherited, the bound endpoints of the parent Executor used to be overridden by those of the child Executor. This meant, if you inherited from Executors but still chose to use the parent Executor in your Flow, a wrong endpoint could have been called. This behavior is fixed by making Executor.requests a nested dict that also includes information about the class name. This helps to properly support Executor inheritance.

    Missing recording logic in connection stub metrics (#5363)

    Recording of request and response size in bytes is fixed to track all cases. This makes these metrics more accurate for the Gateway.

    Move build configs to pyproject (#5351)

    Build requirements have been moved from setup.py to pyproject.toml. This suppresses deprecation warnings that show up when installing Jina.

    New timer should keep labels (#5341)

    The MetricsTimer in instrumentation previously created new timers without keeping the histogram metric labels. This behavior is fixed and new timers retain the same labels.

    Use non-mutable default for MetricsTimer constructor (#5339)

    Use None instead of empty dict as a default value for histogram_metric_labels in MetricsTimer constructor.

    Catch RpcErrors and show better error messages in Client (#5325)

    In the Client, we catch RpcError and show its details instead of showing a standard error message.

    Import OpenTelemetry functions only when tracing is enabled in WorkerRuntime (#5321)

    This release ensures OpenTelemetry functions are only imported when tracing is enabled in the worker.

    ๐Ÿ“— Documentation Improvements

    • Remove 3 off-topic articles
    • Enable flag to convert resource labels to metric labels (#5409)
    • Add reference to Jina Kotlin client from community (#5390)
    • Add section about the transition from DocArray to Jina (#5382)
    • Add tips about supporting multiprocessing with fork in Jina when using macOS (#5379)
    • Fix reference to multiprocessing with spawn section and emphasize the need for entrypoint protection (#5370)
    • Update instructions to build protos locally using protogen Docker image (#5335)
    • Change mentions of JCloud to Jina AI Cloud (#5329)
    • Restructure docs into cloud-native section (#5332)
    • Add contributor acknowledgement and spell checking (#5324)
    • Create a Kubernetes section (#5315)
    • Add reference to Go and PHP clients (#5253)
    • Introduce versioning to the documentation (#5310)
    • Support redirects for removed documentation pages (#5301)
    • Use better Grafana screenshot without random text block (#5306)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Ziniu Yu (@ZiniuYu)
    • Andrei Ungureanu (@Andrei997)
    • Alex Cureton-Griffiths (@alexcg1)
    • Yanlong Wang (@nomagick)
    • Johannes Messner (@JohannesMessner)
    • samsja (@samsja)
    • Joan Fontanals (@JoanFM)
    • Zhaofeng Miao (@mapleeit)
    • Girish Chandrashekar (@girishc13)
    • Jackmin801 (@Jackmin801)
    • Han Xiao (@hanxiao)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Nan Wang (@nan-wang)
    Source code(tar.gz)
    Source code(zip)
  • v3.11.2(Nov 22, 2022)

    Release note

    This release contains 1 hot fix.

    ๐Ÿž Bug Fix

    Unpin jina-hubble-sdk

    To avoid conflicting dependency versions with docarray, unpin the jina-hubble-sdk version dependency

    ๐ŸคŸ Contributors

    Joan Fontanals (@JoanFM )

    Source code(tar.gz)
    Source code(zip)
  • v3.11.0(Oct 24, 2022)

    Release Note (3.11.0)

    Release time: 2022-10-24 14:42:35

    This release contains 6 new features, 1 bug fix and 10 documentation improvements.

    ๐Ÿ†• Features

    Add OpenTelemetry tracing and metrics with basic configuration (#5175)

    Jina now supports OpenTelemetry Tracing and Metrics libraries for increased observability and instrumentation of Jina Runtimes. Read the docs (#5291) for details. We also provide a migration guide for a smooth transition from the previously-supported Prometheus-only based metrics instrumentation.

    You can integrate Jaeger or other distributed tracing tools to collect and visualize request-level and application level service operation attributes. This helps you analyze request-response lifecycle, application behavior and performance.

    from jina import Executor, requests, DocumentArray
    
    class MyExec(Executor):
        @requests
        def encode(self, docs: DocumentArray, **kwargs):
            with self.tracer.start_as_current_span(
                'encode', context=tracing_context
            ) as span:
                with self.monitor(
                    'preprocessing_seconds', 'Time preprocessing the requests'
                ):
                    docs.tensors = preprocessing(docs)
                with self.monitor(
                    'model_inference_seconds', 'Time doing inference the requests'
                ):
                    docs.embedding = model_inference(docs.tensors)
    

    Record existing Prometheus metrics into OpenTelemetry histograms (#5275)

    Prometheus-only based metrics are also available as OpenTelemetry supported metrics. You can switch to OpenTelemetry metrics with little effort.

    Add default tracing interceptors to head gRPC connection pool (#5271)

    Head Runtime supports default traces when forwarding requests to the shard replica.

    Add gRPC metadata to Executors and Gateway deployments (#5221)

    You can now provide HTTP header access tokens as client metadata for accessing external and authentication protected services.

    from jina import Client
    
    client = Client(host='api.clip.jina.ai', port=2096, tls=True, protocol='grpc')
    client.post(on='/encode', metadata=(('authorization', '<your access token>'),))
    

    Show local location of Executors in Hub (#5282)

    Use the list sub command to list the locations of local Hub Executors:

    jina hub list
    

    asciicast

    Dump to Statefulset in K8s when volumes are passed to Executor (#5265)

    Jina will generate a Kubernetes Statefulset manifest for Executors if you provide volumes to the Flow.to_kubernets_yaml() method. Executors can persist data on mounted volumes and correctly scale horizontally if required.

    ๐Ÿž Bug Fixes

    Invalid input raises exception (#5141)

    • Theclient.post() method was not raising an exception if any of the underlying async requests raised an exception.
    • The new version correctly raises the exception raised by any async request. The user can now correctly catch any exceptions and act upon the exception.

    ๐Ÿ“— Documentation Improvements

    • Docs for OpenTelemetry instrumentation (#5291)
    • Add Jina AI Cloud restructure Jcloud and Hub (#5298)
    • Standardize naming conventions (#5285)
    • Fix metrics naming (#5277)
    • Misc clean up (#5255, #5287, #5284, #5290, #5289, #5288)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • AlaeddineAbdessalem (@alaeddine-13)
    • Anthony Le (@AnthonyLe93)
    • felix-wang (@numb3r3)
    • Girish Chandrashekar (@girishc13)
    • zhangkai (@floralatin)
    • Joan Fontanals (@JoanFM)
    • Johannes Messner (@JohannesMessner)
    • samsja (@samsja)
    • Han Xiao (@hanxiao)
    • Alex Cureton-Griffiths (@alexcg1)
    • Andrei Ungureanu (@Andrei997)
    Source code(tar.gz)
    Source code(zip)
  • v3.10.1(Oct 6, 2022)

    Release Note (3.10.1)

    Release time: 2022-10-06 15:05:48

    This release contains 1 new feature, 1 refactor, 3 bug fixes and 7 documentation improvements.

    ๐Ÿ†• Features

    Distributed replicas across different hosts (#5217)

    You can now start different Executors on different machines and use them as replicas by passing their addresses in the host parameter as a list, together with the external flag set to True:

    f.add(host='localhost:12345,91.198.174.192:12346', external=True)
    

    The Flow ensures the requests are load balanced between Executor instances.

    โš™ Refactoring

    Move Hubble related code to hubble-sdk (#5227)

    All code related to hubble and communication with Jina Hub has been moved to the jina-hubble-sdk repo and has been added as an external dependency.

    ๐Ÿž Bug Fixes

    Disable rich traceback in logging (#5242)

    We disabled the use of rich logging when printing tracebacks for Exceptions. This makes it easier to debug issues as the lines are not broken.

    Previously, an Exception would be reported to the console as:

      ERROR  executor0/[email protected] ZeroDivisionError('division by [10/05/22 11:26:23]
           zero')                                                                   
            add "--quiet-error" to suppress the exception                           
           details                                                                  
           โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€ Traceback (most recent call last) โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in process_data                                   โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   179 โ”‚   โ”‚   โ”‚   โ”‚   if self.logger.debug_enable โ”‚                    
           โ”‚   180 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   self._log_data_request( โ”‚                    
           โ”‚   181 โ”‚   โ”‚   โ”‚   โ”‚                               โ”‚                    
           โ”‚ โฑ 182 โ”‚   โ”‚   โ”‚   โ”‚   result = await self._data_r โ”‚                    
           โ”‚   183 โ”‚   โ”‚   โ”‚   โ”‚   if self._successful_request โ”‚                    
           โ”‚   184 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   self._successful_reques โ”‚                    
           โ”‚   185 โ”‚   โ”‚   โ”‚   โ”‚   return result               โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in handle                                         โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   161 โ”‚   โ”‚   )                                   โ”‚                    
           โ”‚   162 โ”‚   โ”‚                                       โ”‚                    
           โ”‚   163 โ”‚   โ”‚   # executor logic                    โ”‚                    
           โ”‚ โฑ 164 โ”‚   โ”‚   return_data = await self._executor. โ”‚                    
           โ”‚   165 โ”‚   โ”‚   โ”‚   req_endpoint=requests[0].header โ”‚                    
           โ”‚   166 โ”‚   โ”‚   โ”‚   docs=docs,                      โ”‚                    
           โ”‚   167 โ”‚   โ”‚   โ”‚   parameters=params,              โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in __acall__                                      โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   290 โ”‚   โ”‚   if req_endpoint in self.requests:   โ”‚                    
           โ”‚   291 โ”‚   โ”‚   โ”‚   return await self.__acall_endpo โ”‚                    
           โ”‚   292 โ”‚   โ”‚   elif __default_endpoint__ in self.r โ”‚                    
           โ”‚ โฑ 293 โ”‚   โ”‚   โ”‚   return await self.__acall_endpo โ”‚                    
           โ”‚   294 โ”‚                                           โ”‚                    
           โ”‚   295 โ”‚   async def __acall_endpoint__(self, req_ โ”‚                    
           โ”‚   296 โ”‚   โ”‚   func = self.requests[req_endpoint]  โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in __acall_endpoint__                             โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   311 โ”‚   โ”‚   โ”‚   if iscoroutinefunction(func):   โ”‚                    
           โ”‚   312 โ”‚   โ”‚   โ”‚   โ”‚   return await func(self, **k โ”‚                    
           โ”‚   313 โ”‚   โ”‚   โ”‚   else:                           โ”‚                    
           โ”‚ โฑ 314 โ”‚   โ”‚   โ”‚   โ”‚   return func(self, **kwargs) โ”‚                    
           โ”‚   315 โ”‚                                           โ”‚                    
           โ”‚   316 โ”‚   @property                               โ”‚                    
           โ”‚   317 โ”‚   def workspace(self) -> Optional[str]:   โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/xxx/Documents/workspace/Jina/jina/jina/seโ€ฆ โ”‚                    
           โ”‚ in arg_wrapper                                    โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   159 โ”‚   โ”‚   โ”‚   โ”‚   def arg_wrapper(            โ”‚                    
           โ”‚   160 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   executor_instance, *arg โ”‚                    
           โ”‚   161 โ”‚   โ”‚   โ”‚   โ”‚   ):  # we need to get the su โ”‚                    
           โ”‚       the self                                    โ”‚                    
           โ”‚ โฑ 162 โ”‚   โ”‚   โ”‚   โ”‚   โ”‚   return fn(executor_inst โ”‚                    
           โ”‚   163 โ”‚   โ”‚   โ”‚   โ”‚                               โ”‚                    
           โ”‚   164 โ”‚   โ”‚   โ”‚   โ”‚   self.fn = arg_wrapper       โ”‚                    
           โ”‚   165                                             โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚ /home/USER/.config/JetBrains/PyCharmCE2022.2/scrโ€ฆ โ”‚                    
           โ”‚ in foo                                            โ”‚                    
           โ”‚                                                   โ”‚                    
           โ”‚   12 โ”‚                                            โ”‚                    
           โ”‚   13 โ”‚   @requests                                โ”‚                    
           โ”‚   14 โ”‚   def foo(self, docs, **kwargs):           โ”‚                    
           โ”‚ โฑ 15 โ”‚   โ”‚   1/0                                  โ”‚                    
           โ”‚   16                                              โ”‚                    
           โ”‚   17 with Flow().add(uses=MyExec) as f:           โ”‚                    
           โ”‚   18                                              โ”‚                    
           โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ                    
           ZeroDivisionError: division by zero                                      
    Traceback (most recent call last):
      File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
        f.search(inputs=[Document()])
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
        return run_async(
      File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
        return asyncio.run(func(*args, **kwargs))
      File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
        return loop.run_until_complete(main)
      File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
        return future.result()
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
        async for resp in c._get_results(*args, **kwargs):
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
        callback_exec(
      File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
        raise BadServer(response.header)
    jina.excepts.BadServer: request_id: "c50a57014fb948ccbd8065ada487a136"
    status {
      code: ERROR
      description: "ZeroDivisionError(\'division by zero\')"
      exception {
        name: "ZeroDivisionError"
        args: "division by zero"
        stacks: "Traceback (most recent call last):\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n    result = await self._data_request_handler.handle(requests=requests)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n    return_data = await self._executor.__acall__(\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n    return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n    return func(self, **kwargs)\n"
        stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n    return fn(executor_instance, *args, **kwargs)\n"
        stacks: "  File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n    1/0\n"
        stacks: "ZeroDivisionError: division by zero\n"
        executor: "MyExec"
      }
    }
    exec_endpoint: "/search"
    target_executor: ""
    
    
    Process finished with exit code 1
    

    After the fix, the error is much clearer:

    ERROR  executor0/[email protected] ZeroDivisionError('division by [10/05/22 11:40:37]
         zero')                                                                   
          add "--quiet-error" to suppress the exception                           
         details                                                                  
         Traceback (most recent call last):                                       
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 182, in process_data                                                
             result = await                                                       
         self._data_request_handler.handle(requests=requests)                     
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 164, in handle                                                      
             return_data = await self._executor.__acall__(                        
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 293, in __acall__                                                   
             return await                                                         
         self.__acall_endpoint__(__default_endpoint__,                            
         **kwargs)                                                                
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 314, in __acall_endpoint__                                          
             return func(self, **kwargs)                                          
           File                                                                   
         "/home/USER/Documents/workspace/Jina/jina/jina/serveโ€ฆ                    
         line 162, in arg_wrapper                                                 
             return fn(executor_instance, *args, **kwargs)                        
           File                                                                   
         "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratcโ€ฆ                    
         line 15, in foo                                                          
             1/0                                                                  
         ZeroDivisionError: division by zero                                      
    Traceback (most recent call last):
    File "/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py", line 19, in <module>
      f.search(inputs=[Document()])
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 271, in post
      return run_async(
    File "/home/USER/Documents/workspace/Jina/jina/jina/helper.py", line 1334, in run_async
      return asyncio.run(func(*args, **kwargs))
    File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/runners.py", line 44, in run
      return loop.run_until_complete(main)
    File "/home/USER/.pyenv/versions/3.9.10/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
      return future.result()
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/mixin.py", line 262, in _get_results
      async for resp in c._get_results(*args, **kwargs):
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/base/grpc.py", line 122, in _get_results
      callback_exec(
    File "/home/USER/Documents/workspace/Jina/jina/jina/clients/helper.py", line 81, in callback_exec
      raise BadServer(response.header)
    jina.excepts.BadServer: request_id: "5d5e0338d0cd4bdd89efc430bbaaa74d"
    status {
    code: ERROR
    description: "ZeroDivisionError(\'division by zero\')"
    exception {
      name: "ZeroDivisionError"
      args: "division by zero"
      stacks: "Traceback (most recent call last):\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/worker/__init__.py\", line 182, in process_data\n    result = await self._data_request_handler.handle(requests=requests)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/runtimes/request_handlers/data_request_handler.py\", line 164, in handle\n    return_data = await self._executor.__acall__(\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 293, in __acall__\n    return await self.__acall_endpoint__(__default_endpoint__, **kwargs)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/__init__.py\", line 314, in __acall_endpoint__\n    return func(self, **kwargs)\n"
      stacks: "  File \"/home/USER/Documents/workspace/Jina/jina/jina/serve/executors/decorators.py\", line 162, in arg_wrapper\n    return fn(executor_instance, *args, **kwargs)\n"
      stacks: "  File \"/home/USER/.config/JetBrains/PyCharmCE2022.2/scratches/scratch_6.py\", line 15, in foo\n    1/0\n"
      stacks: "ZeroDivisionError: division by zero\n"
      executor: "MyExec"
    }
    }
    exec_endpoint: "/search"
    target_executor: ""
    
    
    Process finished with exit code 1
    

    Remove lambda function in logger (#5249)

    We removed a reference to a lambda function in the JinaLogger so that there is no problem pickling it:

    import pickle
    
    from jina import Executor
    
    class MyExecutor(Executor):
    
        def __init__(self, *args, **kwargs):
            super().__init__(*args, **kwargs)
    
    
    executor = MyExecutor()
    
    with open('executor.pickle', 'wb') as f:
        pickle.dump(executor, f)
    

    Previously, an Executor could not be pickled because it had a reference to a JinaLogger.

    This used to raise an AttributeError exception:

    Traceback (most recent call last):
      File "/home/USER/toy.py", line 14, in <module>
        pickle.dump(executor, f)
    AttributeError: Can't pickle local object 'JinaLogger.__init__.<locals>.<lambda>'
    

    Fix gRPC fork support (#5250)

    We enabled proper fork support for gRPC to correctly clean up gRPC usage in parent and child processes. This prevents potential issues when forking Executor processes.

    ๐Ÿ“— Documentation Improvements

    • Explain Executor patching in Kubernetes (#5235)
    • Add note about keeping same Jina versions across microservices (#5239)
    • Add missing request numbers in sample code (#5237)
    • JCloud YAML reorder (#5234)
    • Docs spelling/grammar/punctuation (#5244) (#5240)
    • Fix typos (#5236)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • Sami Jaghouar (@samsja)
    • Alex Cureton-Griffiths (@alexcg1)
    • Johannes Messner (@JohannesMessner)
    • Delgermurun (@delgermurun)
    • AlaeddineAbdessalem (@alaeddine-13)
    • Jackmin801 (@Jackmin801)
    • Joan Fontanals (@JoanFM)
    • Girish Chandrashekar (@girishc13)
    Source code(tar.gz)
    Source code(zip)
  • v3.10.0(Sep 29, 2022)

    Release Note (v3.10.0)

    Release time: 2022-09-29 16:18:19

    This release contains 10 new features, 9 bug fixes and 12 documentation improvements.

    ๐Ÿ†• Features

    Enable jina ping as readiness probe on local and Kubernetes (#5200)

    This feature lets you check the readiness of an entire Flow or its individual components:

    jina ping flow grpc://localhost:12345
    
    jina ping executor localhost:12346
    
    jina ping gateway grpc://localhost:12345
    

    Successful output:

    INFO   [email protected] readiness check succeeded 1 times!!! 
    

    Unsuccessful output:

    This command exits with code 1 when the readiness check is not successful. Therefore, it is a good choice to use as a readiness probe for Executor and Gateway when deployed in Kubernetes.

    Add argument to specify exceptions that will exit the runtime (#5165)

    You can specify which Exceptions should trigger termination of an affected Executor:

    Flow().add(uses=MyExecutor, exit_on_exceptions: ['Exception',  'RuntimeException'])
    

    This ensures an Executor is not stuck in an unusable but technically alive state, and that the Executor can be restarted. This is especially useful when Executor restarts are automated through Kubernetes.

    Retry mechanism for client.post (#5176)

    client.post() accepts max_attempts, initial_backoff, max_backoff and backoff_multiplier parameters to control the capacity to retry requests, when a transient connectivity error occurs, using an exponential backoff strategy:

    client.post(
        on='/',
        inputs=docs,
        max_attempts=5,
        initial_backoff=0.1,
        max_backoff=0.5,
        backoff_multiplier=1.5
    )
    

    Add monitoring of requests size everywhere (#5111)

    The size of requests (jina_request_size_bytes) is now monitored everywhere: On every Executor, every Gateway, and every sharding Head.

    Consequently, this new metric can now be accessed via Prometheus and Grafana.

    JSON logging for JCloud (#5201)

    It is now possible to create logs in JSON format, instead of the standard line-by-line text output.

    Sample output:

    {"created": 1663930387.5850368, "filename": "data_request_handler.py", "funcName": "_load_executor", "levelname": "DEBUG", "lineno": 98, "module": "data_request_handler", "msg": "<jina.serve.executors.BaseExecutor object at 0x7fc1951f78d0> is successfully loaded!", "name": "WorkerRuntime", "pathname": "/home/foo/jina/serve/runtimes/request_handlers/data_request_handler.py", "process": 13386, "processName": "Pod", "thread": 140469860276032, "threadName": "MainThread"}
    

    To enable this, set the environment variable 'JINA_LOG_CONFIG to 'json'.

    Support list-like syntax to specify visible cuda devices (#5187)

    You can now use device id's to specify visible CUDA devices that should be used in the round-robin GPU assignment:

    `CUDA_VISIBLE_DEVICES=RR1,3`
    

    This will assign CUDA devices 1 and 3 to the different Executor replicas in a round-robin fashion.

    Make jina hub push non-blocking (#5129)

    jina hub push is now a non-blocking operation, meaning that the user does not have to wait for the entire push to be completed on the server side.

    While the push is being processed in the background, its status can be checked with the following command:

    jina hub status [<path_to_executor_folder>] [--id TASK_ID] [--verbose] [--replay]
    

    Warning messages in jina hub push (#5156)

    jina hub push can now display helpful warning messages, such as deprecation warnings: Screen Shot 2022-09-09 at 16 32 58

    Add duration info in events (#5157)

    Telemetry data now also includes the duration of every event that is reported.

    ๐Ÿž Bug Fixes

    Fix compatibility with Protobuf Python backend (#5222)

    This bug caused an exception when Python was used as a backend for Protobuf, rather than cpp or udp.

    With this fix, Python can also be used as a backend for Protobuf.

    Move significant params to sequential (#5205)

    This bug (occasionally) caused docker.errors.NullResource: Resource ID was not provided when pulling from Jina Hub. This was due to a faulty local caching implementation, which is now fixed.

    Fix missing secret when logged-in user with --force-update (#5180)

    This bug caused a failure for logged in users when force pushing to Jina Hub: Their secret would not be passed. This is now fixed and force push works as expected.

    Use default gRPC parameters for gRPC connection pool connection (#5211)

    We brought our gRPC client in line with the recommended configuration at https://github.com/grpc/grpc/blob/master/doc/keepalive.md

    Bump DocArray version for new column syntax (#5171)

    Some changes to the column syntax in DocArray required a version bump.

    Increase minimum Protobuf version (#5166)

    Some changes in the Protobuf library required a version bump.

    Remove leftover prints (#5199)

    Some unwanted print statements had sneaked their way into our codebase, and we kicked them right back out!

    ๐Ÿ“— Documentation Improvements

    • Change setenv into environ (#5223)
    • Mention prefetch (#5220)
    • Fix install instructions docker (#5213)
    • Fix grammar, wording, punctuation (#5209)
    • Fix grammar, punctuation (#5208)
    • Clarify exec endpoint usage in http (#5202)
    • Document grpc client limitation (#5193)
    • Update faq and lifetime (#5191)
    • Improve warning about config file in custom docker (#5190)
    • Clarify return behavior in parameters (#5179)
    • Add section for exit_on_exceptions argument (#5172)
    • Labels in flow yaml (#5164)

    ๐ŸคŸ Contributors

    We would like to thank all contributors to this release:

    • AlaeddineAbdessalem (@alaeddine-13)
    • Girish Chandrashekar (@girishc13)
    • Andrei Ungureanu (@Andrei997)
    • Michael Gรผnther (@guenthermi)
    • Deepankar Mahapatro (@deepankarm)
    • Han Xiao (@hanxiao)
    • zhangkai (@floralatin)
    • Joan Fontanals (@JoanFM)
    • Alex Cureton-Griffiths (@alexcg1)
    • Yanlong Wang (@nomagick)
    • Nikolas Pitsillos (@npitsillos)
    • samsja (@samsja)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.3(Sep 29, 2022)

    Release Note (3.9.3)

    Release time: 2022-09-29 16:05:04

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, AlaeddineAbdessalem, Michael Gรผnther, Yanlong Wang, samsja, Alex Cureton-Griffiths, Deepankar Mahapatro, Nikolas Pitsillos, Han Xiao, zhangkai, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [11533962] - add gateway option to jina ping and use ping in readinessProbe (#5200) (Joan Fontanals)
    • [d0838d37] - add multiple attempts options to client.post API (#5176) (Joan Fontanals)
    • [13edf908] - use default grpc parameters for grpc connection pool connection (#5211) (samsja)
    • [216b4bf0] - monitoring: add monitoring of requests size in bytes at all level (#5111) (samsja)
    • [737875f5] - logs: json logging for jcloud (#5201) (Deepankar Mahapatro)
    • [fd4c0347] - support list-like syntax to round robin CUDA devices (#5187) (Joan Fontanals)
    • [9dff4c88] - add duration info in events (#5157) (Joan Fontanals)

    ๐Ÿž Bug fixes

    • [3683fccf] - fix compatibility with protobuf python backend (#5222) (AlaeddineAbdessalem)
    • [e932d6f9] - hubio.fetch_meta: move significant params to sequential (#5205) (Yanlong Wang)
    • [60e9b8de] - remove leftover prints (#5199) (Joan Fontanals)
    • [da186a23] - provide logger and streamer (#5188) (AlaeddineAbdessalem)
    • [7b5c0316] - fix get-openapi-schemas script (#5185) (AlaeddineAbdessalem)
    • [caf4a3d6] - fix missing secret when logged-in user with --force-update and โ€ฆ (#5180) (zhangkai)

    ๐Ÿงผ Code Refactoring

    • [9a6079ef] - separate gateway and asyncio runtime readiness checks (#5224) (AlaeddineAbdessalem)
    • [243639dd] - extract gateway app logic into custom gateway class (#5153) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [147317db] - change setenv into environ (#5223) (Michael Gรผnther)
    • [ad37bdd2] - mention prefetch (#5220) (Joan Fontanals)
    • [b9c8e44e] - fix install instructions docker (#5213) (Joan Fontanals)
    • [7cbf3471] - what-is-jina: fix grammar, wording, punctuation (#5209) (Alex Cureton-Griffiths)
    • [7c7d2cb0] - what-is-modality: fix grammar, punctuation (#5208) (Alex Cureton-Griffiths)
    • [1399e36c] - clarify exec endpoint usage in http (#5202) (Joan Fontanals)
    • [842a585b] - document grpc client limitation (#5193) (AlaeddineAbdessalem)
    • [96c5e7c1] - jcloud: update faq and lifetime (#5191) (Nikolas Pitsillos)
    • [c17e6416] - improve warning about config file in custom docker (#5190) (Joan Fontanals)
    • [22330fcb] - clarify return behavior in parameters (#5179) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [b5977f47] - update contributing to remove labeling (#5225) (Joan Fontanals)
    • [45c7c6e1] - update pull request template link (#5214) (Joan Fontanals)
    • [1eab9ce6] - fix doc template (Han Xiao)
    • [8c2d0758] - docs: update TOC (Jina Dev Bot)
    • [139b068e] - version: the next version will be 3.9.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.2(Sep 15, 2022)

    Release Note (3.9.2)

    Release time: 2022-09-15 14:00:36

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Girish Chandrashekar, zhangkai, AlaeddineAbdessalem, Joan Fontanals, Jina Dev Bot, Andrei Ungureanu, Deepankar Mahapatro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [c51f9014] - hubble async push (#5129) (zhangkai)
    • [3f39ed46] - runtime: add argument to specify exceptions that will exit the runtime (#5165) (Girish Chandrashekar)
    • [27e1f779] - hubio: display warning messages from hubble request (jina hub push) (#5156) (Andrei Ungureanu)

    ๐Ÿž Bug fixes

    • [29ad1750] - exit on exception args only applies to executors (#5169) (AlaeddineAbdessalem)
    • [55188165] - pin docarray version for new column syntax (#5171) (Joan Fontanals)
    • [fa83955c] - increase minimum protobuf version (#5166) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [28aeac8e] - add section for exit_on_exceptions argument (#5172) (Girish Chandrashekar)
    • [33891c46] - jcloud: labels in flow yaml (#5164) (Deepankar Mahapatro)

    ๐Ÿ Unit Test and CICD

    • [b59d0450] - add jina auth token (#5167) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [76dca020] - update announcement in readme and docs (Jina Dev Bot)
    • [f4248ea8] - docs: update TOC (Jina Dev Bot)
    • [fda5623a] - version: the next version will be 3.9.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.1(Sep 8, 2022)

    Release Note (3.9.1)

    Release time: 2022-09-08 14:10:44

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [273fda5a] - merge dryrun into ping (#5151) (Han Xiao)

    ๐Ÿน Other Improvements

    • [0a71009b] - version: the next version will be 3.9.1 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.9.0(Sep 8, 2022)

    Highlights :star2:

    Update to new protobuf version

    We have updated our protobuf dependency to use the latest protobuf version leveraging all the performance improvement that comes with it.

    #5082

    Merge manifest.yml and config.yml for Jina Hub Executors

    We have made it easier for the users to manage their Hub Executors by merging the information from manifest.yml to config.yml which makes the experience much better.

    #5101

    Expose grpc server parameters to Executor

    We expose all the parameters that grpc server can accept for users to be able to configure their services in the most robust and reliable way.

    #5092

    Other changes

    • Add more Dockerfile templates options in jina hub new #5104
    • Add failed and successfull requests metrics #5079

    Bug fixes

    • Fix reading empty list from environment variables #5143
    Source code(tar.gz)
    Source code(zip)
  • v3.8.4(Sep 8, 2022)

    Release Note (3.8.4)

    Release time: 2022-09-08 08:51:32

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Deepankar Mahapatro, AlaeddineAbdessalem, Joan Fontanals, Johannes Messner, Andrei Ungureanu, samsja, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [d1faf6e6] - pass internal flag to telemetry (#5134) (Joan Fontanals)

    ๐Ÿž Bug fixes

    • [e501ddfd] - version: adapt to docker hub v2 (#5146) (Deepankar Mahapatro)
    • [d76e69ac] - fix parsing empty list (#5143) (AlaeddineAbdessalem)
    • [3ec33c08] - gpu dockerfile template include now jina install (#5124) (samsja)
    • [e2967f98] - set minimal docarray dependency (#5133) (Joan Fontanals)
    • [ff803686] - linkerd cd (#5131) (AlaeddineAbdessalem)

    ๐Ÿงผ Code Refactoring

    • [a78612f3] - manifest config (#5101) (Andrei Ungureanu)

    ๐Ÿ Unit Test and CICD

    • [ddf90efb] - remove outdated test (#5139) (Johannes Messner)
    • [a4ea7215] - increase wait time (#5137) (AlaeddineAbdessalem)
    • [478e53f6] - fix k8s tests return responses params (#5127) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [a9b80aec] - docs: update TOC (Jina Dev Bot)
    • [1df9b459] - version: the next version will be 3.8.4 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.3(Sep 5, 2022)

    Release Note (3.8.3)

    Release time: 2022-09-05 14:48:30

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [7b7eecc5] - cap grpc version (#5122) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [47e2cfc3] - docs: update TOC (Jina Dev Bot)
    • [0dbdcaa0] - version: the next version will be 3.8.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.2(Sep 3, 2022)

    Release Note (3.8.2)

    Release time: 2022-09-03 15:56:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Zac Li, samsja, AlaeddineAbdessalem, Jina Dev Bot, tarrantro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [e794c06c] - expose grpc parameters and add production ready keepalive parameters (#5092) (samsja)
    • [6205ffc9] - add gpu dockerfile support to jina hub new (#5104) (AlaeddineAbdessalem)

    ๐Ÿ“— Documentation

    • [9293698b] - rewrite flow add (Han Xiao)
    • [115bc0fa] - rewrite flow add (#5120) (Han Xiao)
    • [22fdbcee] - jcloud: fix jc status link (#5116) (Zac Li)
    • [4dfe6dfb] - jcloud: fix docs for gateway resource customization (#5114) (Zac Li)
    • [6b24bf69] - correct jcloud retention days (#5109) (tarrantro)

    ๐Ÿ Unit Test and CICD

    • [587f6268] - fix test concurrent async flow (#5115) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [763f6773] - update readme (Han Xiao)
    • [2651c506] - disable hub-integration (Han Xiao)
    • [3590c385] - fix readme (Han Xiao)
    • [8303fb25] - docs: improve docs on jcloud k8s (Han Xiao)
    • [a34edc62] - update announcement in readme and docs (Jina Dev Bot)
    • [d124d9af] - docs: update TOC (Jina Dev Bot)
    • [0df7cd8a] - version: the next version will be 3.8.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.1(Aug 30, 2022)

    Release Note (3.8.1)

    Release time: 2022-08-30 19:32:52

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, samsja, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6df36ce5] - client: add profiling function to client and flow (#5105) (Han Xiao)

    ๐Ÿž Bug fixes

    • [78aae025] - fix inconsistent docs and tests on py_modules (#5108) (Han Xiao)
    • [a725010d] - remove flakyness of monitoring integration tests in CI (#5106) (samsja)

    ๐Ÿงผ Code Refactoring

    • [90fa81f3] - move port monitoring to pods parsing (#5100) (samsja)

    ๐Ÿ“— Documentation

    • [95ee283c] - fix naming (#5102) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [d528d947] - fix cd pipeline (Han Xiao)
    • [57625350] - docs: update TOC (Jina Dev Bot)
    • [1ad06216] - docs: fix retention days description (Han Xiao)
    • [9f5f5879] - version: the next version will be 3.8.1 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.8.0(Aug 29, 2022)

    Release Note (3.8.0)

    Release time: 2022-08-29 16:43:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Johannes Messner, tarrantro, Han Xiao, AlaeddineAbdessalem, samsja, Jina Dev Bot, Zac Li, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [527beb85] - bump protobuf version (#5082) (AlaeddineAbdessalem)
    • [c47cb716] - add failed and successful request number metrics (#5079) (AlaeddineAbdessalem)

    ๐Ÿž Bug fixes

    • [c81252ac] - update to protobuf 4.21 new types (#5098) (Johannes Messner)

    ๐Ÿงผ Code Refactoring

    • [6e51e815] - refactor data request handler monitoring (#5088) (samsja)

    ๐Ÿ“— Documentation

    • [b3fdb49c] - correct jcloud retention days (#5096) (tarrantro)
    • [98fb71f1] - jcloud: adjust jcloud monitoring docs(#5081) (Zac Li)
    • [dbea43f3] - remove kong/alb from jcloud document (#5080) (tarrantro)

    ๐Ÿ Unit Test and CICD

    • [27ec3b41] - cleanup pip install (#5094) (AlaeddineAbdessalem)
    • [c0c3bf9a] - downgrade linkerd version in CI (#5090) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [142c0bb1] - docs: add streamline value props (Han Xiao)
    • [af7f6763] - fix readme (Han Xiao)
    • [c39714fa] - docs: add install apple silicon (Han Xiao)
    • [5375b4d3] - update announcement in readme and docs (Jina Dev Bot)
    • [53553b14] - add security report (Han Xiao)
    • [cfe5dd70] - update readme (Han Xiao)
    • [e106796c] - docs: update furo deps (Han Xiao)
    • [079d04b0] - docs: add jcloud to first app (Han Xiao)
    • [64de8608] - docs: add comparing alternatives (Han Xiao)
    • [f9c2d8de] - docs: update TOC (Jina Dev Bot)
    • [2154f98f] - version: the next version will be 3.7.15 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.14(Aug 22, 2022)

    Release Note (3.7.14)

    Release time: 2022-08-22 13:52:20

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿน Other Improvements

    • [4f380c7e] - fix pprint (Han Xiao)
    • [0bd289fe] - docs: update TOC (Jina Dev Bot)
    • [96489914] - version: the next version will be 3.7.14 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.13(Aug 21, 2022)

    Release Note (3.7.13)

    Release time: 2022-08-21 21:03:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Johannes Messner, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [b7bf0ee1] - set workspace in executor init method (#5072) (Johannes Messner)

    ๐Ÿงผ Code Refactoring

    • [f6dcde38] - telemetry (#5078) (Han Xiao)

    ๐Ÿน Other Improvements

    • [8ad0ead5] - docs: refactor create project docs (Han Xiao)
    • [c7e19841] - update readme (Han Xiao)
    • [7b034d34] - docs: refactor telemetry docs (Han Xiao)
    • [76f48f8f] - docs: refactor jcloud docs (Han Xiao)
    • [e506da44] - docs: refactor hub docs (Han Xiao)
    • [b91f2084] - docs: refactor executors docs (Han Xiao)
    • [f8a63ccf] - docs: refactor flow basic docs (Han Xiao)
    • [885914ad] - docs: restructure docs (Han Xiao)
    • [36fdebe2] - docs: fix docs in jcloud (Han Xiao)
    • [4673e7ab] - docs: update TOC (Jina Dev Bot)
    • [a4c48d1b] - version: the next version will be 3.7.13 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.12(Aug 18, 2022)

    Release Note (3.7.12)

    Release time: 2022-08-18 22:23:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, AlaeddineAbdessalem, Johannes Messner, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6f87524c] - include jcloud cli (#5074) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [10c63db2] - update pr template with reminder to add a description (#5062) (Johannes Messner)
    • [6fbd2be9] - docs: update TOC (Jina Dev Bot)
    • [7d82fdc3] - version: the next version will be 3.7.12 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.11(Aug 17, 2022)

    Release Note (3.7.11)

    Release time: 2022-08-17 21:09:48

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [15fd60d5] - cli: jina auth cli error (Han Xiao)

    ๐Ÿน Other Improvements

    • [80393316] - version: the next version will be 3.7.11 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.10(Aug 17, 2022)

    Release Note (3.7.10)

    Release time: 2022-08-17 12:22:49

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, tarrantro, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [b9bddd22] - cli: jina auth cli error (#5070) (Han Xiao)

    ๐Ÿ“— Documentation

    • [840c221f] - add gateway timeout and update shared gpu (#5069) (tarrantro)

    ๐Ÿน Other Improvements

    • [bffe263c] - docs: update TOC (Jina Dev Bot)
    • [94c83cdb] - version: the next version will be 3.7.10 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.9(Aug 16, 2022)

    Release Note (3.7.9)

    Release time: 2022-08-16 16:25:11

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, AlaeddineAbdessalem, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    ๐Ÿงผ Code Refactoring

    • [c1c500aa] - remove unused hub environment variables (#5067) (AlaeddineAbdessalem)

    ๐Ÿน Other Improvements

    • [442a86cd] - version: the next version will be 3.7.9 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.8(Aug 16, 2022)

    Release Note (3.7.8)

    Release time: 2022-08-16 15:18:50

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, Jina Dev Bot, Zac Li, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [bf864748] - hub: use hubble sdk (#5064) (Han Xiao)

    ๐Ÿ“— Documentation

    • [317e9ce1] - jcloud: doc for gateway customization (#5065) (Zac Li)

    ๐Ÿน Other Improvements

    • [a6be9481] - update announcement in readme and docs (Jina Dev Bot)
    • [f3bcdaa8] - docs: update TOC (Jina Dev Bot)
    • [dfd1e07d] - docs: fix typos (Han Xiao)
    • [d579cb63] - version: the next version will be 3.7.8 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.7(Aug 15, 2022)

    Release Note (3.7.7)

    Release time: 2022-08-15 18:14:16

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Johannes Messner, Han Xiao, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [22563074] - cache hub pull with build env (#5061) (Johannes Messner)

    ๐Ÿน Other Improvements

    • [e7377947] - docs: fix typos (Han Xiao)
    • [6d218fbf] - version: the next version will be 3.7.7 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.6(Aug 15, 2022)

    Release Note (3.7.6)

    Release time: 2022-08-15 14:04:25

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Han Xiao, tarrantro, Zac Li, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿงผ Code Refactoring

    • [49a842d9] - unify temp path to dot cache jina (#5058) (Han Xiao)

    ๐Ÿ“— Documentation

    • [731d492c] - add CPU and GPU type (#5060) (tarrantro)
    • [d8f6f418] - rewrite the telemetry session (#5059) (Han Xiao)
    • [01111539] - jcloud: autoscale docs (#5056) (Zac Li)

    ๐Ÿน Other Improvements

    • [299a908b] - docs: add what is jina (Han Xiao)
    • [23914a81] - add what is cross multi modal (Han Xiao)
    • [0c78b993] - docs: update TOC (Jina Dev Bot)
    • [102bc6de] - version: the next version will be 3.7.6 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.5(Aug 10, 2022)

    Release Note (3.7.5)

    Release time: 2022-08-10 10:19:23

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Jina Dev Bot, zhangkai, samsja, Deepankar Mahapatro, tarrantro, ๐Ÿ™‡

    ๐Ÿ†• New Features

    • [6cc4497a] - hubble support env variables (#5023) (zhangkai)
    • [12404535] - add dryrun to cli (#5050) (samsja)

    ๐Ÿ“— Documentation

    • [1d60655d] - jcloud: fix formatting (#5048) (Deepankar Mahapatro)
    • [6df83cf1] - improve grpc third party client documentation (#5047) (samsja)
    • [8c6321ac] - refactor resources, add gpu, kong (#5027) (tarrantro)

    ๐Ÿน Other Improvements

    • [d6d4c30a] - update announcement in readme and docs (Jina Dev Bot)
    • [9fee2dff] - docs: update TOC (Jina Dev Bot)
    • [fe7b8894] - version: the next version will be 3.7.5 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.4(Aug 3, 2022)

    Release Note (3.7.4)

    Release time: 2022-08-03 12:31:57

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [4c3d760f] - do not send target_executor to Executors (#5041) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [a51b42cc] - update announcement in readme and docs (Jina Dev Bot)
    • [ecb95829] - docs: update TOC (Jina Dev Bot)
    • [286bdce8] - version: the next version will be 3.7.4 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.3(Jul 28, 2022)

    Release Note (3.7.3)

    Release time: 2022-07-28 14:29:23

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Joan Fontanals, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [032bd5e6] - fix specific params problem with branches (#5038) (Joan Fontanals)

    ๐Ÿน Other Improvements

    • [81999e97] - version: the next version will be 3.7.3 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
  • v3.7.2(Jul 28, 2022)

    Release Note (3.7.2)

    Release time: 2022-07-28 09:19:06

    ๐Ÿ™‡ We'd like to thank all contributors for this new release! In particular, Zac Li, Jina Dev Bot, ๐Ÿ™‡

    ๐Ÿž Bug fixes

    • [fb88015e] - add more logging in grpc dry run (#5036) (Zac Li)

    ๐Ÿน Other Improvements

    • [5116b922] - docs: update TOC (Jina Dev Bot)
    • [32dd47af] - version: the next version will be 3.7.2 (Jina Dev Bot)
    Source code(tar.gz)
    Source code(zip)
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
Deal or No Deal? End-to-End Learning for Negotiation Dialogues

Introduction This is a PyTorch implementation of the following research papers: (1) Hierarchical Text Generation and Planning for Strategic Dialogue (

Facebook Research 1.4k Dec 29, 2022
hashily is a Python module that provides a variety of text decoding and encoding operations.

hashily is a python module that performs a variety of text decoding and encoding functions. It also various functions for encrypting and decrypting text using various ciphers.

DevMysT 5 Jul 17, 2022
Enterprise Scale NLP with Hugging Face & SageMaker Workshop series

Workshop: Enterprise-Scale NLP with Hugging Face & Amazon SageMaker Earlier this year we announced a strategic collaboration with Amazon to make it ea

Philipp Schmid 161 Dec 16, 2022
A python script that will use hydra to get user and password to login to ssh, ftp, and telnet

Hydra-Auto-Hack A python script that will use hydra to get user and password to login to ssh, ftp, and telnet Project Description This python script w

2 Jan 16, 2022
A Pytorch implementation of "Splitter: Learning Node Representations that Capture Multiple Social Contexts" (WWW 2019).

Splitter โ €โ € A PyTorch implementation of Splitter: Learning Node Representations that Capture Multiple Social Contexts (WWW 2019). Abstract Recent inte

Benedek Rozemberczki 201 Nov 09, 2022
CPC-big and k-means clustering for zero-resource speech processing

The CPC-big model and k-means checkpoints used in Analyzing Speaker Information in Self-Supervised Models to Improve Zero-Resource Speech Processing.

Benjamin van Niekerk 5 Nov 23, 2022
Code for paper "Which Training Methods for GANs do actually Converge? (ICML 2018)"

GAN stability This repository contains the experiments in the supplementary material for the paper Which Training Methods for GANs do actually Converg

Lars Mescheder 884 Nov 11, 2022
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

ALBERT ***************New March 28, 2020 *************** Add a colab tutorial to run fine-tuning for GLUE datasets. ***************New January 7, 2020

Google Research 3k Dec 26, 2022
Deduplication is the task to combine different representations of the same real world entity.

Deduplication is the task to combine different representations of the same real world entity. This package implements deduplication using active learning. Active learning allows for rapid training wi

63 Nov 17, 2022
iSTFTNet : Fast and Lightweight Mel-spectrogram Vocoder Incorporating Inverse Short-time Fourier Transform

iSTFTNet : Fast and Lightweight Mel-spectrogram Vocoder Incorporating Inverse Short-time Fourier Transform This repo try to implement iSTFTNet : Fast

Rishikesh (เค‹เคทเคฟเค•เฅ‡เคถ) 126 Jan 02, 2023
A simple implementation of N-gram language model.

About A simple implementation of N-gram language model. Requirements numpy Data preparation Corpus Training data for the N-gram model, a text file lik

4 Nov 24, 2021
Utility for Google Text-To-Speech batch audio files generator. Ideal for prompt files creation with Google voices for application in offline IVRs

Google Text-To-Speech Batch Prompt File Maker Are you in the need of IVR prompts, but you have no voice actors? Let Google talk your prompts like a pr

Ponchotitlรกn 1 Aug 19, 2021
An end to end ASR Transformer model training repo

END TO END ASR TRANSFORMER ๆœฌ้กน็›ฎๅŸบไบŽtransformer 6*encoder+6*decoder็š„ๅŸบๆœฌ็ป“ๆž„ๆž„้€ ็š„็ซฏๅˆฐ็ซฏ็š„่ฏญ้Ÿณ่ฏ†ๅˆซ็ณป็ปŸ Model Instructions 1.ๆ•ฐๆฎๅ‡†ๅค‡: ่‡ช่กŒไธ‹่ฝฝๆ•ฐๆฎ๏ผŒ้ตๅพชๆ–‡ไปถ็ป“ๆž„ๅฆ‚ไธ‹๏ผš โ”œโ”€โ”€ data โ”‚ โ”œโ”€โ”€ train โ”‚

ๆ—ท่ง†ๅคฉๅ…ƒ MegEngine 10 Jul 19, 2022
Conditional Transformer Language Model for Controllable Generation

CTRL - A Conditional Transformer Language Model for Controllable Generation Authors: Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong,

Salesforce 1.7k Dec 28, 2022
The (extremely) naive sentiment classification function based on NBSVM trained on wisesight_sentiment

thai_sentiment The naive sentiment classification function based on NBSVM trained on wisesight_sentiment เธงเธดเธ˜เธตเธ•เธดเธ”เธ•เธฑเน‰เธ‡ pip install thai_sentiment==0.1.3

Charin 7 Dec 08, 2022
๐Ÿค— The largest hub of ready-to-use NLP datasets for ML models with fast, easy-to-use and efficient data manipulation tools

๐Ÿค— The largest hub of ready-to-use NLP datasets for ML models with fast, easy-to-use and efficient data manipulation tools

Hugging Face 15k Jan 02, 2023
A repo for open resources & information for people to succeed in PhD in CS & career in AI / NLP

A repo for open resources & information for people to succeed in PhD in CS & career in AI / NLP

420 Dec 28, 2022
A 30000+ Chinese MRC dataset - Delta Reading Comprehension Dataset

Delta Reading Comprehension Dataset ๅฐ้”้–ฑ่ฎ€็†่งฃ่ณ‡ๆ–™้›† Delta Reading Comprehension Dataset (DRCD) ๅฑฌๆ–ผ้€š็”จ้ ˜ๅŸŸ็น้ซ”ไธญๆ–‡ๆฉŸๅ™จ้–ฑ่ฎ€็†่งฃ่ณ‡ๆ–™้›†ใ€‚ ๆœฌ่ณ‡ๆ–™้›†ๆœŸๆœ›ๆˆ็‚บ้ฉ็”จๆ–ผ้ท็งปๅญธ็ฟ’ไน‹ๆจ™ๆบ–ไธญๆ–‡้–ฑ่ฎ€็†่งฃ่ณ‡ๆ–™้›†ใ€‚ ๆœฌ่ณ‡ๆ–™้›†ๅพž2,108็ฏ‡

272 Dec 15, 2022
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference

DeeBERT This is the code base for the paper DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference. Code in this repository is also available

Castorini 132 Nov 14, 2022
The swas programming language

The Swas programming language This is a language that was made for fun. Installation Step 0: Make sure you have python installed Step 1. Clone this re

Swas.py 19 Jul 18, 2022