needlestack.indices package

Submodules

needlestack.indices.faiss_indices module

needlestack.indices.index module

class needlestack.indices.index.BaseIndex[source]

Bases: object

Base class for index implementations. Defines interfaces for populating data and performing kNN queries.

add_vectors(X, metadatas)[source]

Add the vectors to existing index

property count

Number of vectors in the vector space

Return type

int

property dimension

Spatial dimensions for the vector space

Return type

int

static from_proto(proto)[source]

Factory method to construct the correct implementation of a BaseIndex from a protobuf. Specific index types are imported in this function so their dependent packages do not need to be installed

Parameters

proto (BaseIndex) – Protobuf defining how to load data for the index

Return type

BaseIndex

get_vector_and_metadata(id)[source]

Returns the vector and metadata for a particular item id

Parameters

id (str) – ID within metadata

Return type

Tuple[ndarray, Metadata]

Returns an array of distances and index ids

Parameters

  • X (ndarray) – Matrix of vectors to perform kNN search

  • k (int) – Number of neighbors

Return type

Tuple[ndarray, ndarray]

load()[source]

Load data into memory

modified_time = None
populate(data)[source]

Populate BaseIndex from dictionary

Parameters

data (Dict[~KT, ~VT]) – Dictionary of key, value pairs for attributes

populate_from_proto(proto)[source]

Populate BaseIndex from protobuf defining data source

Parameters

proto – Protobuf with data on how to populate a particular BaseIndex implementation

query(X, k)[source]

Returns a list of list of knn query results. Each result is a tuple of (distance, metadata) pairs.

Parameters

  • X (ndarray) – Matrix of vectors to perform kNN search for

  • k (int) – Number of neighbors

Return type

List[List[SearchResultItem]]

retrieve(id)[source]
Return type

RetrievalResultItem

serialize()[source]

Serialize the current index to a protobuf

set_vectors(X, metadatas)[source]

Set the vectors for this index

update_available()[source]

Data source has an update available

Module contents