# Rounding parameter from float to int for quantization

**URL:** <https://discuss.tvm.apache.org/t/rounding-parameter-from-float-to-int-for-quantization/11479>\
**Category:** RFC\
**Created:** [November 16, 2021, 8:49am UTC](https://discuss.tvm.apache.org/t/rounding-parameter-from-float-to-int-for-quantization/11479 "2021-11-16T08:49:32Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![elvin-n](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/elvin-n/32/2682_2.png) [@elvin-n](https://discuss.tvm.apache.org/u/elvin-n)\
**Post date:** [November 16, 2021, 8:49am UTC](https://discuss.tvm.apache.org/t/rounding-parameter-from-float-to-int-for-quantization/11479/1 "2021-11-16T08:49:32Z")

</div>

We have not fully aligned and not consistent behaviour in regard of float rounding to int values for quantization.

- qnn.op.quantize does not have round param and uses tir.round that rounds to nearest
- qnn.op.requantize has round param which might accept two values “UPWARD” (default) and "TONEAREST.

UPWARD that TVM has is something very artificial that might have sense for quantization and improving accuracy but is not described in standard, is not supported by hardware and is not used by any deep learning frameworks. Having this TVM “UPWARD” currently as default parameter, requires to implement more actions and leads to performance implications.

**Proposal:**

1. Remove round parameter from qnn.op.requantize at all
2. Or standardize the rounding in TVM, add it to qnn.op.quantize, make default value that something is supported by hardware natively like “TONEAREST” or “FLOOR”

_More details:_ The current TVM requantize UPWARD mode is defined follow way:

```auto
    rounding : string, optional
        Defines the rounding direction when the value is midway between two
        representable values.

```

I.e. it affect only numbers having x.5 values, especial for negative numbers. It is not matched to any round standard in spite of [Rounding - Wikipedia](https://en.wikipedia.org/wiki/Rounding) describes many approaches. TVM has new special one.

How requantize is called from TVM importers:

- mxnet: calls 4 times, no passing of rounding param, i.e. default “UPWARD” is used
- onnx: calls twice, once without rounding, second time with rounding == “TONEAREST”
- qnn\_torch: one call without rounding param passing
- tflite: 9 calls without rounding param passing

If we take a look how rounding happens/defined in frameworks, then in most cases it is not standardized, only onnx defines explicitly. Other can use different round strategy for different implementations depending on hardware

- onnx   
from [https://github.com/onnx/onnx/blob/master/docs/Operators.md#QuantizeLinear](https://github.com/onnx/onnx/blob/master/docs/Operators.md#QuantizeLinear)  
For (x / y\_scale), it’s rounding to nearest ties to even. Refer to [Rounding - Wikipedia](https://en.wikipedia.org/wiki/Rounding) for details.
- mxnet   

  - no mention of the rounding in API, mxnet requantize op does not have round param   
  - cpu - intgemm is used ([https://github.com/kpu/intgemm](https://github.com/kpu/intgemm)), for rounding - \_mm256\_floor\_ps

- qnn-torch   

  - std::lrintf is used in fused\_nbit\_rowwise\_conversion.cc/FloatToFused8BitRowwiseQuantized\_\_base   
the default value of rounding returned by std::fegetround is 0 that stands for FE\_TONEAREST
  - floorf is used pytorch/caffe2/quantization/server/norm\_minimization\_avx2.cc

- tflite   

  - cl, metal - uses round, i.e. to nearest   
  - cpu - gemmlowp::RoundingDivideByPOT, rounding-to-nearest [https://github.com/google/gemmlowp/blob/master/fixedpoint/fixedpoint.h#L365](https://github.com/google/gemmlowp/blob/master/fixedpoint/fixedpoint.h#L365)