Convert a collection of features to a fixed-dimensional matrix using the hashing trick.

Overview

FeatureHasher

Convert a collection of features to a fixed-dimensional matrix using the hashing trick.

Note, this requires Jina>=2.2.4.

Example

Here I use FeatureHasher to hash each sentence of Pride and Prejudice into a 128-dim vector, and then use .match to find top-K similar sentences.

from jina import Document, DocumentArray, Flow

# load 
   
d = Document(uri='https://www.gutenberg.org/files/1342/1342-0.txt').convert_uri_to_text()

# cut into non-empty sentences store in a DA
da = DocumentArray(Document(text=s.strip()) for s in d.text.split('\n') if s.strip())

# use FeatureHasher in a Flow
f = Flow().add(uses='jinahub://FeatureHasher')

embed_da = DocumentArray()
with f:
    f.post('/', da, on_done=lambda req: embed_da.extend(req.docs), show_progress=True)

print('self-matching...')
embed_da.match(embed_da, exclude_self=True, limit=5, normalization=(1, 0))
print('total sentences: ', len(embed_da))
for d in embed_da:
    print(d.text)
    for m in d.matches:
        print(m.scores['cosine'], m.text)
    input()
           [email protected][I]:🎉 Flow is ready to use!
	🔗 Protocol: 		GRPC
	🏠 Local access:	0.0.0.0:52628
	🔒 Private network:	192.168.178.31:52628
	🌐 Public address:	217.70.138.123:52628
⠹       DONE ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:01 100% ETA: 0 seconds 40 steps done in 1 second
total sentences:  12153
The Project Gutenberg eBook of Pride and Prejudice, by Jane Austen

   
     *** END OF THE PROJECT GUTENBERG EBOOK PRIDE AND PREJUDICE ***

    
      *** START OF THE PROJECT GUTENBERG EBOOK PRIDE AND PREJUDICE ***

     
       production, promotion and distribution of Project Gutenberg-tm

      
        Pride and Prejudice

       
         By Jane Austen This eBook is for the use of anyone anywhere in the United States and 
        
          This eBook is for the use of anyone anywhere in the United States and 
         
           by the awkwardness of the application, and at length wholly 
          
            Elizabeth passed the chief of the night in her sister’s room, and 
           
             the happiest memories in the world. Nothing of the past was 
            
              charities and charitable donations in all 50 states of the United 
            
           
          
         
        
       
      
     
    
   

In practice, you can implement matching and storing via an indexer inside Flow. This example is only for demo purpose so any non-feature hashing related ops are implemented outside the Flow to avoid distraction.

Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
This is a Cryptographied Password Manager, a tool for storing Passwords in a Secure way

Cryptographied Password Manager This is a Cryptographied Password Manager, a tool for storing Passwords in a Secure way without using external Service

Francesco 3 Nov 23, 2022
Orthrus is a macOS agent that uses Apple's MDM to backdoor a device using a malicious profile.

Orthrus is a macOS agent that uses Apple's MDM to backdoor a device using a malicious profile. It effectively runs its own MDM server and allows the operator to interface with it using Mythic.

Mythic Agents 37 Dec 06, 2022
Port scanning tool that uses Python3. Created by Noble Wilson

Hello There! My name is Noble Wilson and I am an aspiring IT/InfoSec coder practicing for my future. ________________________________________________

1 Nov 23, 2021
A Modified version of TCC's Osprey poc framework......

fierce-fish fierce-fish是由TCC(斗象能力中心)出品并维护的开源漏洞检测框架osprey的改写,去掉臃肿功能的精简版本poc框架 PS:真的用不惯其它臃肿的功能,不过作为一个收集漏洞poc && exp的框架还是非常不错的!!! osprey For beginners fr

lUc1f3r11 10 Dec 30, 2022
Reusable Lightweight Pythonic Dependency Injection Library

Vacuna Inject everything! Vacuna is a little library to provide dependency management for your python code. Install pip install vacuna Usage import va

Fernando Martínez González 16 Sep 15, 2021
M.E.A.T. - Mobile Evidence Acquisition Toolkit

M.E.A.T. - Mobile Evidence Acquisition Toolkit Meet M.E.A.T! From Jack Farley - BlackStone Discovery This toolkit aims to help forensicators perform d

1 Nov 11, 2021
Monty Hall Problem simulation written in Python.

Monty Hall Problem Simulation monty_hall_sim is a brute-force method of determining the optimal strategy for the Monty Hall Problem. Usage Set boolean

Xavier D 1 Aug 29, 2022
The Linux defender anti-virus software ported to work on CentOS Linux.

By: Seanpm2001, Et; Al. Top README.md Read this article in a different language Sorted by: A-Z Sorting options unavailable ( af Afrikaans Afrikaans |

Sean P. Myrick V19.1.7.2 2 Sep 12, 2022
A simple password generator using Python Tkinter.

Password-Generator-using-Python A simple password generator that generates password for you. User can Copy the password to Clipboard. Project made usi

Prashant Agheda 1 Nov 02, 2022
This enforces signatures for CVE-2021-44228 across all policies on a BIG-IP ASM device

f5-waf-enforce-sigs-CVE-2021-44228 This enforces signatures for CVE-2021-44228 across all policies on a BIG-IP ASM device Overview This script enforce

Ismael Gonçalves 5 Mar 31, 2022
Official implementation of the paper "Backdoor Attacks on Self-Supervised Learning".

SSL-Backdoor Abstract Large-scale unlabeled data has allowed recent progress in self-supervised learning methods that learn rich visual representation

UMBC Vision 44 Nov 21, 2022
This is a keylogger in python for Windows, Mac and Linux!

Python-Keylogger This is a keylogger in python for Windows, Mac and Linux! #How to use it by downloading the zip file? Download the zip file first The

Zeus_Dxvxm 2 Nov 12, 2021
Lazarus analysis tools and research report

Lazarus Research This repository publishes analysis reports and analysis tools for Operation Dream Job and Operation JTrack for Lazarus. Tools Python

JPCERT Coordination Center 50 Sep 13, 2022
test application for the licence key web app.

licence_software_test_app Make sure you set your database values in a .env file to the folder. Install MYSQL connector: pip install mysql-connector-py

Carl Beattie 1 Oct 28, 2021
Tool for finding PHP source code vulnerabilities.

vulnz Tool for finding php source code vulnerabilities. Scans PHP source code and prints out potentially dangerous lines. This tool is useful for secu

Mateo Hanžek 1 Jan 14, 2022
CVE-2022-22965 : about spring core rce

CVE-2022-22965: Spring-Core-Rce EXP 特性: 漏洞探测(不写入 webshell,简单字符串输出) 自定义写入 webshell 文件名称及路径 不会追加写入到同一文件中,每次检测写入到不同名称 webshell 文件 支持写入 冰蝎 webshell 代理支持,可

东方有鱼名为咸 53 Nov 09, 2022
Security system to prevent Shoulder Surfing Attacks

Surf_Sec Security system to prevent Shoulder Surfing Attacks. REQUIREMENTS: Python 3.6+ XAMPP INSTALLED METHOD TO CONFIGURE PROJECT: Clone the repo to

Aman Anand 1 Jan 27, 2022
Sentinel-1 SAR time series analysis for OSINT use

SARveillance Sentinel-1 SAR time series analysis for OSINT use. Description Generates a time lapse GIF of the Sentinel-1 satellite images for the loca

21 Dec 09, 2022
LinOTP - the open source solution for two factor authentication

LinOTP LinOTP - the Open Source solution for multi-factor authentication Copyright © 2010-2019 KeyIdentity GmbH Coypright © 2019- arxes-tolina GmbH In

LinOTP 462 Jan 02, 2023
对naabu的端口扫描结果,调用nmap进行指纹识别

naabu2nmap 对naabu的端口扫描结果,调用nmap进行指纹识别

Se7en 12 Nov 22, 2022