# How to indicate LLVM use \`VQMOVN.s16\`

**URL:** <https://discuss.tvm.apache.org/t/how-to-indicate-llvm-use-vqmovn-s16/9682>\
**Category:** Uncategorized\
**Created:** [April 13, 2021, 9:48am UTC](https://discuss.tvm.apache.org/t/how-to-indicate-llvm-use-vqmovn-s16/9682 "2021-04-13T09:48:53Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![fantasyRqg](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/fantasyrqg/32/2566_2.png) [@fantasyRqg](https://discuss.tvm.apache.org/u/fantasyRqg)\
**Post date:** [April 13, 2021, 9:48am UTC](https://discuss.tvm.apache.org/t/how-to-indicate-llvm-use-vqmovn-s16/9682/1 "2021-04-13T09:48:53Z")

</div>

`Clamp Op` not indicate LLVM generate `VQMOVN.s16 `

```python
    dtype = 'int8'

    M = 224
    N = 224
    # Algorithm
    A = te.placeholder((M, M), name="A", dtype=dtype)
    B = te.placeholder((M, N), name="B", dtype=dtype)

    C = te.compute((M, N), lambda x, y: (A[x, y].astype('int16') * B[x, y].astype('int16')), name="C")
    C = topi.clip(C, tvm.tir.min_value(dtype).value, tvm.tir.max_value(dtype).value).astype(dtype)
    # C = C.astype(dtype)

    s = te.create_schedule(C.op)
    print(tvm.lower(s, [A, B], simple_mode=True))
    func = tvm.build(s, [A, B, C], target=target, name="scale_one")
    print(func.get_source('asm'))

```

generated assmebly code:

```auto
	vld1.64	{d20, d21}, [r1:128]
	vmin.s16	q10, q10, q8
	vmax.s16	q10, q10, q9
	vst1.64	{d20, d21}, [r1:128]

```

---

<div class="post-metadata">

**Author:** ![FrozenGene](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/frozengene/32/1114_2.png) [@FrozenGene](https://discuss.tvm.apache.org/u/FrozenGene)\
**Post date:** [April 13, 2021, 11:24am UTC](https://discuss.tvm.apache.org/t/how-to-indicate-llvm-use-vqmovn-s16/9682/2 "2021-04-13T11:24:56Z")

</div>

> [@fantasyRqg](#):
>
> ```auto
> dtype = 'int8'
> 
> M = 224
> N = 224
> # Algorithm
> A = te.placeholder((M, M), name="A", dtype=dtype)
> B = te.placeholder((M, N), name="B", dtype=dtype)
> 
> C = te.compute((M, N), lambda x, y: (A[x, y].astype('int16') * B[x, y].astype('int16')), name="C")
> C = topi.clip(C, tvm.tir.min_value(dtype).value, tvm.tir.max_value(dtype).value).astype(dtype)
> # C = C.astype(dtype)
> 
> s = te.create_schedule(C.op)
> print(tvm.lower(s, [A, B], simple_mode=True))
> func = tvm.build(s, [A, B, C], target=target, name="scale_one")
> print(func.get_source('asm'))
> 
> ```

Because `clip`’s definition is

```python
    def _compute(*indices):
        value = x(*indices)
        const_min = tvm.tir.const(a_min, value.dtype)
        const_max = tvm.tir.const(a_max, value.dtype)
        return tvm.te.max(tvm.te.min(value, const_max), const_min)

```

so we generate `vmax` and `vmin` from `max` and `min`

However, the latest LLVM could optimize it. I mean LLVM 11. You could add one flag `-mattr=+mve`. I test it. We could generate `VQMOVN` instruction correctly, but you should make sure your arm cpu could support this instruction.

---

<div class="post-metadata">

**Author:** ![fantasyRqg](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/fantasyrqg/32/2566_2.png) [@fantasyRqg](https://discuss.tvm.apache.org/u/fantasyRqg)\
**Post date:** [April 15, 2021, 4:14am UTC](https://discuss.tvm.apache.org/t/how-to-indicate-llvm-use-vqmovn-s16/9682/3 "2021-04-15T04:14:37Z")

</div>

With flag `-mattr=+mve` generate `VQMOVNB.S16` instead of `VQMOVN.S16`.

`VQMOVNB.S16` is not supported by my target device.

My compile target define:

```python
target_t = 'llvm -device=arm_cpu -model=bcm2835 -mtriple=armv7a-linux-gnueabihf -mattr=+neon,+mve'

```

`--------------------------- split line -----------------------`

My main goal is efficiently execute quantized model.  
` print(tvm.lower(s, [A, B], simple_mode=True))` output:

```auto
primfn(A_1: handle, B_1: handle) -> ()
  attr = {"global_symbol": "main", "tir.noalias": True}
  buffers = {B: Buffer(B_2: Pointer(int8), int8, [224, 224], []),
             A: Buffer(A_2: Pointer(int8), int8, [224, 224], [])}
  buffer_map = {A_1: A, B_1: B} {
  attr [C: Pointer(int16)] "storage_scope" = "global";
  allocate(C, int16, [50176]);
  attr [compute: Pointer(int8)] "storage_scope" = "global";
  allocate(compute, int8, [50176]) {
    for (x: int32, 0, 224) {
      for (y: int32, 0, 224) {
        C[((x*224) + y)] = (cast(int16, (int8*)A_2[((x*224) + y)])*cast(int16, (int8*)B_2[((x*224) + y)]))
      }
    }
    for (i0: int32, 0, 224) {
      for (i1: int32, 0, 224) {
        C[((i0*224) + i1)] = max(min((int16*)C[((i0*224) + i1)], 127i16), -128i16)
      }
    }
    for (i0_1: int32, 0, 224) {
      for (i1_1: int32, 0, 224) {
        compute[((i0_1*224) + i1_1)] = cast(int8, (int16*)C[((i0_1*224) + i1_1)])
      }
    }
  }
}

```

Generated assembly code with `VQMOVNB.S16` still have three loops. It allocates a large buffer for temporary data.

How to merge three loop to one?

Assembly code I want:

```auto
	vld1.8 {d16}, [r9:64], r1
	vld1.8 {d18}, [r4:64], r1
	vmull.s8 q1, d18, d16
	vqmovn.s16 d18, q1
	vst1.8 {d18}, [r5:64], lr

```

---

<div class="post-metadata">

**Author:** ![ramana-arm](https://discuss.tvm.apache.org/letter_avatar_proxy/v4/letter/r/ecb155/32.png) [@ramana-arm](https://discuss.tvm.apache.org/u/ramana-arm)\
**Post date:** [April 15, 2021, 8:24am UTC](https://discuss.tvm.apache.org/t/how-to-indicate-llvm-use-vqmovn-s16/9682/4 "2021-04-15T08:24:17Z")

</div>

I will point out that MVE and Neon will never be present on the same CPU implementation, using them together in the same output object file and indeed in the same function is not correct.

MVE is suitable for the M profile instruction set of the AArch32 ISA i.e. micro-controllers while Neon or Advanced SIMD is suitable for the A profile variant of the AArch32 ISA.

Ramana

---

<div class="post-metadata">

**Author:** ![fantasyRqg](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/fantasyrqg/32/2566_2.png) [@fantasyRqg](https://discuss.tvm.apache.org/u/fantasyRqg)\
**Post date:** [April 15, 2021, 8:49am UTC](https://discuss.tvm.apache.org/t/how-to-indicate-llvm-use-vqmovn-s16/9682/5 "2021-04-15T08:49:51Z")

</div>

Thanks for correct me.

Where could I find full documentation of LLVM `neon`, `mve` attrs.  
I have searched countless time, still no clue.

---

<div class="post-metadata">

**Author:** ![fantasyRqg](https://discuss.tvm.apache.org/user_avatar/discuss.tvm.apache.org/fantasyrqg/32/2566_2.png) [@fantasyRqg](https://discuss.tvm.apache.org/u/fantasyRqg)\
**Post date:** [April 21, 2021, 10:16am UTC](https://discuss.tvm.apache.org/t/how-to-indicate-llvm-use-vqmovn-s16/9682/6 "2021-04-21T10:16:03Z")

</div>

Finally figure out working solution.  
Modify LLVM to support optimize `truncate(smax(smin(a,b),c) ---> vqmovns(c)`

- [Working LLVM repo](https://github.com/fantasyRqg/llvm-project/commit/d7a24469cf88067a7323c11e2b6391d48885e985)
- [LLVM Phabricator commit](https://reviews.llvm.org/D100938)

tvm python script:

```python
    dtype = 'uint8'
    ddtype = 'uint16'

    M = 224
    N = 224
    # Algorithm
    A = te.placeholder((M, M), name="A", dtype=dtype)
    B = te.placeholder((M, N), name="B", dtype=dtype)
    min_v = tir.min_value(dtype).value
    max_v = tir.max_value(dtype).value
    C = te.compute((M, N), lambda x, y: A[x, y].astype(ddtype) * B[x, y].astype(ddtype), name="C")

    C_clip = topi.clip(C, min_v, max_v)
    C_int8 = C_clip.astype(dtype)

    s = te.create_schedule(C_int8.op)
    s[C].compute_inline()
    s[C_clip].compute_inline()

    yo, yi = s[C_int8].split(C_int8.op.axis[1], 8)
    s[C_int8].vectorize(yi)

    print(tvm.lower(s, [A, B, C_int8], simple_mode=True))
    target_t = 'llvm -device=arm_cpu -model=bcm2835 -mtriple=armv7a-linux-gnueabihf -mattr=+neon'

    with tvm.transform.PassContext(opt_level=3):
        func = tvm.build(s, [A, B, C_int8], target=target_t, name="scale_one")

    print(func.get_source('asm'))

```
