Thanks for bringing this up. VTA is a specialized accelerator that have certain restrictions(only work with 8bit fix point) and restricted ALU. As a result, the models we can feed to it is somewhat restricted to quantized models.
There is an ongoing effort on bringing automatic model quantizer to map a relay program to VTA compatible quantized models. Once that is checked in, we can have a path from TF models