# unity **URL:** https://discuss.tvm.apache.org/c/development/unity/14.md [Latest](https://discuss.tvm.apache.org/latest.md) · [Categories](https://discuss.tvm.apache.org/categories.md) --- ## [About the unity category](https://discuss.tvm.apache.org/t/about-the-unity-category/14245)
**Author:** [@tqchen](https://discuss.tvm.apache.org/u/tqchen)\ **Replies:** 0
--- ## [\[RFC\] Annotate Custom Scope layout Relax pass for Adreno GPU](https://discuss.tvm.apache.org/t/rfc-annotate-custom-scope-layout-relax-pass-for-adreno-gpu/18052)
**Author:** [@srkreddy1238](https://discuss.tvm.apache.org/u/srkreddy1238)\ **Replies:** 5\ **Last updated:** [February 26, 2025, 5:29pm UTC](https://discuss.tvm.apache.org/t/rfc-annotate-custom-scope-layout-relax-pass-for-adreno-gpu/18052 "2025-02-26T17:29:10Z")
Objectives: The default layout for any Tensor should be Texture. We plan to introduce Reduction, GeneralReduction and FallBack schedules if an operator has a texture argument. Exceptions possible like if the tensor doe… --- ## [\[Discussion\] A Technical Approach to LLMs with TVM Unity](https://discuss.tvm.apache.org/t/discussion-a-technical-approach-to-llms-with-tvm-unity/15171)
**Author:** [@Hzfengsy](https://discuss.tvm.apache.org/u/Hzfengsy)\ **Replies:** 16\ **Last updated:** [July 26, 2024, 3:01am UTC](https://discuss.tvm.apache.org/t/discussion-a-technical-approach-to-llms-with-tvm-unity/15171 "2024-07-26T03:01:48Z")
hi, community, Large language models (LLMs) have been revolutionizing our world and enhancing our lives. We build the project MLC-LLM, trying to bring LLMs to everyone’s local devices. It’s one of our vertical applicat… --- ## [Relax shape refinement implements](https://discuss.tvm.apache.org/t/relax-shape-refinement-implements/17154)
**Author:** [@psunn](https://discuss.tvm.apache.org/u/psunn)\ **Replies:** 3\ **Last updated:** [April 26, 2024, 11:29am UTC](https://discuss.tvm.apache.org/t/relax-shape-refinement-implements/17154 "2024-04-26T11:29:44Z")
In Relax design notes, one of the key constructs for symbolic shape computations is match\_shape. According to the design notes and early demos, match\_shape was originally defined as a Relax expression R.match\_shape. (I c… --- ## [Relax IR, fields that require SeqExpr](https://discuss.tvm.apache.org/t/relax-ir-fields-that-require-seqexpr/17055)
**Author:** [@Lunderberg](https://discuss.tvm.apache.org/u/Lunderberg)\ **Replies:** 5\ **Last updated:** [April 19, 2024, 2:30pm UTC](https://discuss.tvm.apache.org/t/relax-ir-fields-that-require-seqexpr/17055 "2024-04-19T14:30:17Z")
The Relax IR requires the FunctionNode::body, IfNode::true\_branch, and IfNode::false\_branch to be instances of relax::SeqExpr. If these Relax requirements are violated, correctly-implemented transformations may raise an … --- ## [\[Unity\] \[Frontend\] Failed to load Range op](https://discuss.tvm.apache.org/t/unity-frontend-failed-to-load-range-op/16532)
**Author:** [@anders](https://discuss.tvm.apache.org/u/anders)\ **Replies:** 2\ **Last updated:** [March 20, 2024, 8:19am UTC](https://discuss.tvm.apache.org/t/unity-frontend-failed-to-load-range-op/16532 "2024-03-20T08:19:51Z")
I use Relax onnx frontend to load onnx model with only one Range op. the model is simple, and I will upload it later. after loaded the onnx model, I got the follow IRModule, but failed to build with relax @I.ir\_mod… --- ## [InplaceTransformParams](https://discuss.tvm.apache.org/t/inplacetransformparams/16323)
**Author:** [@tqchen](https://discuss.tvm.apache.org/u/tqchen)\ **Replies:** 5\ **Last updated:** [February 27, 2024, 5:26pm UTC](https://discuss.tvm.apache.org/t/inplacetransformparams/16323 "2024-02-27T17:26:53Z")
Background The transform\_params pass allows us to lift the original weight pre-transforms into a separate function, so we can transform the layout of the paramters. class Before: @R.function def main(x: R.Tensor… --- ## [\[RFC\]\[Unity\]\[Relax\]DistIR Compilation Pipeline](https://discuss.tvm.apache.org/t/rfc-unity-relax-distir-compilation-pipeline/15600)
**Author:** [@DiTian](https://discuss.tvm.apache.org/u/DiTian)\ **Replies:** 3\ **Last updated:** [February 22, 2024, 2:50pm UTC](https://discuss.tvm.apache.org/t/rfc-unity-relax-distir-compilation-pipeline/15600 "2024-02-22T14:50:01Z")
Intruduction In this document, we propose a design to lower DistIR, a logical unified distirbuted IR, to ranked physical single-device Relax IRs that is ready to be consumed by the current TVM compiler infrastructure. Af… --- ## [Tensorize when intrinsics doesn't fit perfectly with tensor shapes](https://discuss.tvm.apache.org/t/tensorize-when-intrinsics-doesnt-fit-perfectly-with-tensor-shapes/15075)
**Author:** [@slai-nick](https://discuss.tvm.apache.org/u/slai-nick)\ **Replies:** 4\ **Last updated:** [February 20, 2024, 11:59am UTC](https://discuss.tvm.apache.org/t/tensorize-when-intrinsics-doesnt-fit-perfectly-with-tensor-shapes/15075 "2024-02-20T11:59:34Z")
Working on integration for a custom tensor accelerator with BYOC. So far I have understood how I can tensorize the matrix core operation, input/weights reading, output writing in the case where shapes are heavenly divis… --- ## [\[DISCUSS\] Inplace and Fusion Opportunities in LLM Runtime](https://discuss.tvm.apache.org/t/discuss-inplace-and-fusion-opportunities-in-llm-runtime/16292)
**Author:** [@tqchen](https://discuss.tvm.apache.org/u/tqchen)\ **Replies:** 5\ **Last updated:** [January 30, 2024, 3:26am UTC](https://discuss.tvm.apache.org/t/discuss-inplace-and-fusion-opportunities-in-llm-runtime/16292 "2024-01-30T03:26:59Z")
Inplace and Fusion Opportunities This is a quick note about what opportunities we can enable through inplace support. While inplace operations do not necessarily bring better speedup, there are quite a bit opportunities … --- ## [Could tir transform support cache read to different dtype, like float16 to int32?](https://discuss.tvm.apache.org/t/could-tir-transform-support-cache-read-to-different-dtype-like-float16-to-int32/16260)
**Author:** [@leiwen83](https://discuss.tvm.apache.org/u/leiwen83)\ **Replies:** 2\ **Last updated:** [January 19, 2024, 2:32am UTC](https://discuss.tvm.apache.org/t/could-tir-transform-support-cache-read-to-different-dtype-like-float16-to-int32/16260 "2024-01-19T02:32:47Z")
For some case, we may need to read in like float16, and coalesce into longer type like int32 for better performance. But current cache\_read API seem directly use the original block’s dtype, so how to tell tir try diffe… --- ## [\[Unity\] PrimValue and 0-d tensors in Relax functions](https://discuss.tvm.apache.org/t/unity-primvalue-and-0-d-tensors-in-relax-functions/16214)
**Author:** [@Lunderberg](https://discuss.tvm.apache.org/u/Lunderberg)\ **Replies:** 4\ **Last updated:** [January 11, 2024, 2:59pm UTC](https://discuss.tvm.apache.org/t/unity-primvalue-and-0-d-tensors-in-relax-functions/16214 "2024-01-11T14:59:04Z")
During a conversation with @slyubomirsky, it came up that Relax functions have two distinct ways to express scalar values. Both PrimValue (struct info of R.Prim) and and scalar tensors (struct info of R.Tensor(ndim=0)) … --- ## [\[ISSUE\] Instruction does not dominate all uses](https://discuss.tvm.apache.org/t/issue-instruction-does-not-dominate-all-uses/16208)
**Author:** [@anders](https://discuss.tvm.apache.org/u/anders)\ **Replies:** 2\ **Last updated:** [January 8, 2024, 3:02am UTC](https://discuss.tvm.apache.org/t/issue-instruction-does-not-dominate-all-uses/16208 "2024-01-08T03:02:53Z")
" Instruction does not dominate all uses" is not a new issue since in unity branch, more strict check has merged in codegen\_llvm.cc(see following this-\>Verify()): tvm/src/target/llvm/codegen\_llvm.cc at main · apache/tvm … --- ## [Unity AOT Support?](https://discuss.tvm.apache.org/t/unity-aot-support/16173)
**Author:** [@nskaplan](https://discuss.tvm.apache.org/u/nskaplan)\ **Replies:** 2\ **Last updated:** [December 30, 2023, 4:58pm UTC](https://discuss.tvm.apache.org/t/unity-aot-support/16173 "2023-12-30T16:58:55Z")
I see in the unity RFC that AOT execution is a goal. From looking at the unity branch, it doesn’t seem like this is implemented yet-- I don’t see any c codegens in src/relax/backend/contrib and I don’t see AOTExecutor re… --- ## [Recursive visiting in \`PyExprMutator\`](https://discuss.tvm.apache.org/t/recursive-visiting-in-pyexprmutator/15224)
**Author:** [@kparzysz](https://discuss.tvm.apache.org/u/kparzysz)\ **Replies:** 5\ **Last updated:** [December 18, 2023, 7:13pm UTC](https://discuss.tvm.apache.org/t/recursive-visiting-in-pyexprmutator/15224 "2023-12-18T19:13:54Z")
I was trying to do something like this, basically have my own visitor that will do some work on its own and use the superclass’s visitor to do the rest: @relax.expr\_functor.mutator class MyMutator(relax.PyExprMutator): … --- ## [\[Unity\] Redundant argument in the tir.Buffer.vload()](https://discuss.tvm.apache.org/t/unity-redundant-argument-in-the-tir-buffer-vload/16142)
**Author:** [@anders](https://discuss.tvm.apache.org/u/anders)\ **Replies:** 1\ **Last updated:** [December 15, 2023, 8:01am UTC](https://discuss.tvm.apache.org/t/unity-redundant-argument-in-the-tir-buffer-vload/16142 "2023-12-15T08:01:21Z")
tvm/python/tvm/tir/buffer.py at unity · apache/tvm (github.com) in tir/buffer.py, Buffer.vload(begin, dtype=None) function, the dtype is redundant. as no testing case has used the dtype arg, and according to the .cc imp… --- ## [How to reclaim the memory in the relax function output, so that it would not oom?](https://discuss.tvm.apache.org/t/how-to-reclaim-the-memory-in-the-relax-function-output-so-that-it-would-not-oom/16083)
**Author:** [@leiwen83](https://discuss.tvm.apache.org/u/leiwen83)\ **Replies:** 2\ **Last updated:** [December 4, 2023, 2:36am UTC](https://discuss.tvm.apache.org/t/how-to-reclaim-the-memory-in-the-relax-function-output-so-that-it-would-not-oom/16083 "2023-12-04T02:36:58Z")
Hi, I current use PackedFunc directly to get the model infered result like “xx=vm-\>GetFunction(“yy”); out = xx()”. Things goes well, except the output would continuously alloc new memory, which at last cause oom. I thi… --- ## [\[DISCUSS\] Default Compilation Flow for Scan and Sort](https://discuss.tvm.apache.org/t/discuss-default-compilation-flow-for-scan-and-sort/15949)
**Author:** [@tqchen](https://discuss.tvm.apache.org/u/tqchen)\ **Replies:** 14\ **Last updated:** [November 29, 2023, 5:52am UTC](https://discuss.tvm.apache.org/t/discuss-default-compilation-flow-for-scan-and-sort/15949 "2023-11-29T05:52:25Z")
This posts originates from original items here https://github.com/apache/tvm/issues/15851 It would be great for us to discuss our strategy for implementing operations like scan(cumsum) and sort. Both of these operators … --- ## [Logic Errors in PrimFunc without strict Check](https://discuss.tvm.apache.org/t/logic-errors-in-primfunc-without-strict-check/15988)
**Author:** [@anders](https://discuss.tvm.apache.org/u/anders)\ **Replies:** 5\ **Last updated:** [November 17, 2023, 6:27am UTC](https://discuss.tvm.apache.org/t/logic-errors-in-primfunc-without-strict-check/15988 "2023-11-17T06:27:59Z")
Many time, when I use relax.build() API to deploy a AI Model, Error occur in tvm::codegen::CodeGenLLVM::Verify() after tvm::codegen::CodeGenLLVM::Finish(). there are generally two type errors in LLVM Side: a):operator ar… --- ## [\[bubble up\] Logic Errors in PrimFunc without strict Check](https://discuss.tvm.apache.org/t/bubble-up-logic-errors-in-primfunc-without-strict-check/16001)
**Author:** [@anders](https://discuss.tvm.apache.org/u/anders)\ **Replies:** 0\ **Last updated:** [November 17, 2023, 2:35am UTC](https://discuss.tvm.apache.org/t/bubble-up-logic-errors-in-primfunc-without-strict-check/16001 "2023-11-17T02:35:53Z")
bubble up for this issue :kissing\_heart: --- ## [ONNX Quantize Ops Status](https://discuss.tvm.apache.org/t/onnx-quantize-ops-status/15987)
**Author:** [@brian-pieces](https://discuss.tvm.apache.org/u/brian-pieces)\ **Replies:** 0\ **Last updated:** [November 15, 2023, 10:53pm UTC](https://discuss.tvm.apache.org/t/onnx-quantize-ops-status/15987 "2023-11-15T22:53:25Z")
I tried using the Relax ONNX frontend to convert an ONNX model with a QuantizeLinear layer but it failed saying this layer isn’t supported yet. I was wondering what the status of quantized ops is in Unity? I did a bit o… --- ## [\[Unity\] Open Development Meeting, Nov. 21](https://discuss.tvm.apache.org/t/unity-open-development-meeting-nov-21/15986)
**Author:** [@slyubomirsky](https://discuss.tvm.apache.org/u/slyubomirsky)\ **Replies:** 0\ **Last updated:** [November 15, 2023, 9:40pm UTC](https://discuss.tvm.apache.org/t/unity-open-development-meeting-nov-21/15986 "2023-11-15T21:40:53Z")
Our next meeting will take place at 11:30am PST on Nov. 14, 2023. The agenda for the meeting will be as follows: Progress on website or Youtube channel for community meetings by your loving hosts (@slyubomirsky and @y… --- ## [\[Unity\] Open Dev Meeting Nov. 14, 2023](https://discuss.tvm.apache.org/t/unity-open-dev-meeting-nov-14-2023/15973)
**Author:** [@slyubomirsky](https://discuss.tvm.apache.org/u/slyubomirsky)\ **Replies:** 1\ **Last updated:** [November 15, 2023, 3:17pm UTC](https://discuss.tvm.apache.org/t/unity-open-dev-meeting-nov-14-2023/15973 "2023-11-15T15:17:07Z")
Our next meeting will take place at 5:00pm PST on Nov. 14, 2023. Please note that @yongwww and I will be taking over hosting from now on, and welcome other volunteers who would like to host meetings (let’s thank @psrivas… --- ## [Common Subexpr elimination pass replaces constant args with vars](https://discuss.tvm.apache.org/t/common-subexpr-elimination-pass-replaces-constant-args-with-vars/15971)
**Author:** [@sanirudh](https://discuss.tvm.apache.org/u/sanirudh)\ **Replies:** 2\ **Last updated:** [November 14, 2023, 9:47am UTC](https://discuss.tvm.apache.org/t/common-subexpr-elimination-pass-replaces-constant-args-with-vars/15971 "2023-11-14T09:47:53Z")
Hi all, I see a scenario where the Common Subexpr Elimination pass replaces constant arguments with vars and I wanted to ask if it would make sense to modify the pass to avoid working on constants. This works like the o… --- ## [\[Relax Frontend\] An Pull Requests for Squeeze OpConverter](https://discuss.tvm.apache.org/t/relax-frontend-an-pull-requests-for-squeeze-opconverter/15934)
**Author:** [@anders](https://discuss.tvm.apache.org/u/anders)\ **Replies:** 2\ **Last updated:** [November 4, 2023, 3:36am UTC](https://discuss.tvm.apache.org/t/relax-frontend-an-pull-requests-for-squeeze-opconverter/15934 "2023-11-04T03:36:01Z")
While deploying the bert model with relax frontend，I found a bug in Squeeze OpConvert，we could directly fetch the axis info if there are in attr argument. the PR is here: https://github.com/apache/tvm/pull/16059 --- ## [Int32/Int64 issue when codegen into llvm::Function?](https://discuss.tvm.apache.org/t/int32-int64-issue-when-codegen-into-llvm-function/15915)
**Author:** [@anders](https://discuss.tvm.apache.org/u/anders)\ **Replies:** 8\ **Last updated:** [November 2, 2023, 11:34am UTC](https://discuss.tvm.apache.org/t/int32-int64-issue-when-codegen-into-llvm-function/15915 "2023-11-02T11:34:10Z")
I construct a simple IRModule only with Relax function like follow: @I.ir\_module class Module: @R.function def main(x: R.Tensor(("m",), dtype="float32"), y: R.Tensor(("m",), dtype="float32")) -\> … --- ## [Validity of Common Subexpression elimination pass that simplifies call\_tir args](https://discuss.tvm.apache.org/t/validity-of-common-subexpression-elimination-pass-that-simplifies-call-tir-args/15832)
**Author:** [@sanirudh](https://discuss.tvm.apache.org/u/sanirudh)\ **Replies:** 3\ **Last updated:** [October 24, 2023, 1:37am UTC](https://discuss.tvm.apache.org/t/validity-of-common-subexpression-elimination-pass-that-simplifies-call-tir-args/15832 "2023-10-24T01:37:57Z")
Hi all, When EliminateCommonSubexpr pass runs on an IR where there are 2 call\_tir nodes with same arguments, it replaces the tuple with a var. Before Common Subexpression Elimination: a = R.call\_tir(cls.strided\_slice,… --- ## [Better memory planning for relax?](https://discuss.tvm.apache.org/t/better-memory-planning-for-relax/15749)
**Author:** [@leiwen83](https://discuss.tvm.apache.org/u/leiwen83)\ **Replies:** 3\ **Last updated:** [September 26, 2023, 1:55pm UTC](https://discuss.tvm.apache.org/t/better-memory-planning-for-relax/15749 "2023-09-26T13:55:26Z")
Hi, I am current try to make tvm running with big resolution image, so I want to internal memory could be reused whenever possible. When I get thing work with relax, I find current relax memory plan still far from perf… --- ## [Question regarding Relax pattern matcher](https://discuss.tvm.apache.org/t/question-regarding-relax-pattern-matcher/15739)
**Author:** [@rutkoor](https://discuss.tvm.apache.org/u/rutkoor)\ **Replies:** 5\ **Last updated:** [September 26, 2023, 4:30am UTC](https://discuss.tvm.apache.org/t/question-regarding-relax-pattern-matcher/15739 "2023-09-26T04:30:06Z")
Hello, I was working on Relax IR module and trying to optimize certain patterns. I am going through relax pattern matcher examples and listed down below questions. pre and post ops information will be available in the… --- ## [\[Unity\]\[Tutorial\] TVM Unity BYOC](https://discuss.tvm.apache.org/t/unity-tutorial-tvm-unity-byoc/14561)
**Author:** [@sunggg](https://discuss.tvm.apache.org/u/sunggg)\ **Replies:** 7\ **Last updated:** [September 18, 2023, 11:57pm UTC](https://discuss.tvm.apache.org/t/unity-tutorial-tvm-unity-byoc/14561 "2023-09-18T23:57:13Z")
As we presented in TVMCon’23. in TVM Unity, compilation flow is modular and composable including BYOC. This tutorial walk-throughs how BYOC offloading works in Unity pass infra and how it works with other passes, such a… [Next page](https://discuss.tvm.apache.org/c/development/unity/14.md?page=1)