Skip to content

Error when requesting a very large number of responses #3

@ravimody

Description

@ravimody

get_similar('user', user, 'product', 100000000)
returns

all2vec/__init__.py in get_similar(self, entity_type, entity_id, match_type, num_similar, oversample, normalize)
    168         match_vector = self.get_vector(entity_type, entity_id)
    169         return self.get_similar_vector(
--> 170             match_vector, match_type, num_similar, oversample, normalize)
    171
    172     def get_scores_vector(self, vector, match_type, match_id_array, normalize):

all2vec/__init__.py in get_similar_vector(self, match_vector, match_type, num_similar, oversample, normalize)
    156
    157         similar_items = self._annoy_objects[match_type].get_nns_by_vector(
--> 158             match_vector, num_similar, search_k)
    159         # compute inner products, and sort
    160         scores = self.get_scores_vector(

all2vec/__init__.py in get_nns_by_vector(self, vec, n, search_k)
     67         """Get nearest neighbors from an input vector."""
     68         nns = self._ann_obj.get_nns_by_vector(vec, n, search_k)
---> 69         return [self._ann_map_inv[x] for x in nns]
     70
     71     def get_item_vector(self, entity_id):

TypeError: 'NoneType' object is not iterable

Digging deeper, it looks like all2vec is not properly capturing this ANNOY error

> u.get_nns_by_vector([1] * num_dimensions, 100000000, search_k = 100000000 * 100)
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
OverflowError: signed integer is greater than maximum

This error will come up whenever n or search_k is larger than the maximum integer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions