Converting TF2 object detection models with use_combined_nms=true

Questions
#1

I’m trying to compile TF2 object detection models from TF2 model zoo ssd_mobilenet_v2_320x320_coco17_tpu-8 I followed the steps in https://github.com/tensorflow/models/pull/9707 to freeze the TF2 model by modifying ssd_mobilenet_v2_320x320_coco17_tpu-8/pipeline.config

post_processing {
  batch_non_max_suppression {
    score_threshold: 9.99999993922529e-09
    iou_threshold: 0.6000000238418579
    max_detections_per_class: 100
    max_total_detections: 100
    use_static_shapes: false
    change_coordinate_frame: false
    use_combined_nms: true
  }
  score_converter: SIGMOID
}

and exported using exporter_main_v2.py

python object_detection/exporter_main_v2.py
–input_type=float_image_tensor
–pipeline_config_path=ssd_mobilenet_v2_320x320_coco17_tpu-8/pipeline.config
–trained_checkpoint_dir=ssd_mobilenet_v2_320x320_coco17_tpu-8/checkpoint
–output_directory=output/ssd_mobilenet_v2_320x320_coco17_tpu-8_float_batchN_nms

I try to convert this saved_model to relay using

import tensorflow as tf
from tensorflow.python.framework.convert_to_constants import convert_variables_to_constants_v2

import tvm
from tvm.relay.frontend.tensorflow2 import from_tensorflow

import sys

imported = tf.saved_model.load(sys.argv[1])
ff = imported.signatures['serving_default']
ff = convert_variables_to_constants_v2(ff)
graph_def = ff.graph.as_graph_def(True)

mod, params = from_tensorflow(graph_def)

But it fails when converting to relay with the error: TVMError: In function ir.TensorType(0: Array, 1: DataType) → relay.TensorType: error while converting argument 1: [09:49:32] /workspace/tvm/include/tvm/runtime/data_type.h:374: unknown type variant

Does tvm support TF2 object detection models with use_combined_nms=true, if so is there a tutorial or steps that I can follow and make sure I can compile these models using tvm?

#2

It is recommended to convert the exported tf2 model to ONNX, and use our ONNX frontend. Unfortunately, models produced by exporter_main_v2.py are generally not good fit for deployment (it adds image preprocessing loop even if there is only one image, and NMS is unrolled per class).

I have some models and scripts in https://github.com/masahi/tf2-detection-to-tvm, the models there were generated after modifying tensorflow/models a bit, and converted to ONNX. In particular, checkout https://github.com/masahi/tf2-detection-to-tvm/blob/master/ssd_mobilenet/test_ssd_mobilenet_combined_nms.py.

#3

I’ve written a detailed guide for how to export TF2 detection models before, I posted it at tf2-od-export.md · GitHub

#4

Thanks for the information and detailed guide @masahi. I’ll follow the instructions in the guide to compile TF2 models.