Skip to content

TFLite MobileNet quantized (int8) fails with OpenCV preprocessing for MobileNet (rgbf32) #40

@psyhtest

Description

@psyhtest

Running MobileNet quantized with TFLite on images preprocessed with OpenCV for MobileNet into rgbf32 fails:

$ ck benchmark program:image-classification-tflite --speed --skip_print_timers \
--repetitions=1 --env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=10 \
--dep_add_tags.images=preprocessed,using-opencv,normalized-for.mobilenet \
--dep_add_tags.weights=mobilenet,quantized
...
Summary:
-------------------------------
Graph loaded in 0.001913s
All images loaded in 0.002327s
All images classified in 0.460941s
Average classification time: 0.043231s
Accuracy top 1: 0.0 (0 of 10)
Accuracy top 5: 0.0 (0 of 10)
--------------------------------

The same works fine with TF-C++:

$ ck benchmark program:image-classification-tf-cpp --speed --skip_print_timers \
--repetitions=1 --env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=10 \
--dep_add_tags.images=preprocessed,using-opencv,normalized-for.mobilenet \
--dep_add_tags.weights=mobilenet,quantized
...
Summary:
-------------------------------
Graph loaded in 0.081466s
All images loaded in 0.002623s
All images classified in 0.528722s
Average classification time: 0.011366s
Accuracy top 1: 0.7 (7 of 10)
Accuracy top 5: 0.9 (9 of 10)
--------------------------------

or with TFLite on images preprocessed using OpenCV into rgb8:

$ ck benchmark program:image-classification-tflite --speed --skip_print_timers \
--repetitions=1 --env.CK_BATCH_SIZE=1 --env.CK_BATCH_COUNT=10 \
--dep_add_tags.images=preprocessed,using-opencv,rgb8 \
--dep_add_tags.weights=mobilenet,quantized
...
Summary:
-------------------------------
Graph loaded in 0.001789s
All images loaded in 0.000663s
All images classified in 0.477427s
Average classification time: 0.043434s
Accuracy top 1: 0.7 (7 of 10)
Accuracy top 5: 0.9 (9 of 10)
--------------------------------

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions