A containerized REST API around OpenAI's CLIP model.

Last update: Nov 06, 2022

Overview

OpenAI's CLIP — REST API

This is a container wrapping OpenAI's CLIP model in a RESTful interface.

Running the container locally

First, build the container:

docker build -t clip-container:latest .

Then, you can run it:

docker run -it -p 8080:8080 --name "clip-container" --rm clip-container:latest /opt/ml/code/serve

Sending requests:

The container exposes two different endpoints:

GET /ping: Returns 200 status if the container is working properly.
POST /invocations: Processes a list of images and returns the list of labels with their corresponding probabilities.

Here is an example request assuming the container is listening in port 8080:

curl --location --request POST 'http://localhost:8080/invocations' \
--header 'Content-Type: application/json' \
--data-raw '{
    "images": [
        "https://images.unsplash.com/photo-1597308680537-1ba44407ffc0?ixid=MXwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHw%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=1834&q=80",
        "https://images.unsplash.com/photo-1589270216117-7972b3082c7d?ixid=MXwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHw%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=1834&q=80"],
    "classes": ["person", "bag", "person with a bag", "woman riding a horse", "woman with a bag", "woman with black shirt and a bag"]
}'

The response looks like this:

[
    {
        "url": "https://images.unsplash.com/photo-1597308680537-1ba44407ffc0?ixid=MXwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHw%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=1834&q=80", 
        "labels": [
            "woman with black shirt and a bag", 
            "woman with a bag", 
            "person with a bag", 
            "bag", "person"
        ], 
        "probs": [1.0, 1.7488513970320696e-09, 1.1663764917350243e-19, 4.179975909038141e-30, 3.77612043676229e-30]
    }, 
    {
        "url": "https://images.unsplash.com/photo-1589270216117-7972b3082c7d?ixid=MXwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHw%3D&ixlib=rb-1.2.1&auto=format&fit=crop&w=1834&q=80", 
        "labels": [
            "person with a bag", 
            "woman with black shirt and a bag", 
            "bag", 
            "woman with a bag", 
            "person"
        ], 
        "probs": [1.0, 2.4879632576357835e-08, 2.065714813830402e-13, 7.658033346455602e-15, 1.1307645811408335e-23]
    }
]

SageMaker Integration

This container is compatible with SageMaker so you should be able to host it as a SageMaker endpoint with no modifications. The code supports GPU and CPU instances.

A containerized REST API around OpenAI's CLIP model.

Related tags

Overview

OpenAI's CLIP — REST API

Running the container locally

Sending requests:

SageMaker Integration

Owner

Santiago Valdarrama

[NeurIPS 2021]: Are Transformers More Robust Than CNNs? (Pytorch implementation & checkpoints)

Vehicle detection using machine learning and computer vision techniques for Udacity's Self-Driving Car Engineer Nanodegree.

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit

vit for few-shot classification

CarND-LaneLines-P1 - Lane Finding Project for Self-Driving Car ND

Exploration of some patients clinical variables.

This repository contains the database and code used in the paper Embedding Arithmetic for Text-driven Image Transformation

Pytorch Implementation of Continual Learning With Filter Atom Swapping (ICLR'22 Spolight) Paper

Submodular Subset Selection for Active Domain Adaptation (ICCV 2021)

Do Neural Networks for Segmentation Understand Insideness?

Scale-aware Automatic Augmentation for Object Detection (CVPR 2021)

MQBench: Towards Reproducible and Deployable Model Quantization Benchmark

Source code for Acorn, the precision farming rover by Twisted Fields

PyZebrascope - an open-source Python platform for brain-wide neural activity imaging in behaving zebrafish

这是一个yolo3-tf2的源码，可以用于训练自己的模型。

Official implementation of CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification

LoFTR:Detector-Free Local Feature Matching with Transformers CVPR 2021

Attack on Confidence Estimation algorithm from the paper "Disrupting Deep Uncertainty Estimation Without Harming Accuracy"

Narya API allows you track soccer player from camera inputs, and evaluate them with an Expected Discounted Goal (EDG) Agent

Repository for the paper : Meta-FDMixup: Cross-Domain Few-Shot Learning Guided byLabeled Target Data