[RFC] tlcpack: Thirdparty Binary Packages

Development pre-RFC
#6

Understood. Thanks @tqchen!

#7

The wheels are built on a newer version of CentOS. Pip wheel for CPU is manylinux2010 compatibility and wheels for CUDA are manylinux2014 compatibility. Releasing wheels with different CUDA versions is to accommodate different develop and deploy environment.

2 Likes
#8

BTW, @haichen shall we also release runtime-only version of tlcpack? It would be helpful if people want to bake TVM-compiled operators in other frameworks shallowly in python

#9

Thanks for your work.

I tested the wheels however I got CUDA error with tlcpack_cu102-0.7.dev1-cp36-cp36m-manylinux2014_x86_64.whl:

 tvm._ffi.base.TVMError: Traceback (most recent call last):
  [bt] (3) /usr/local/lib/python3.6/dist-packages/tvm/libtvm.so(TVMFuncCall+0x48) [0x7fe7e6f0dc48]
  [bt] (2) /usr/local/lib/python3.6/dist-packages/tvm/libtvm.so(std::_Function_handler<void (tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*), tvm::runtime::detail::PackFuncVoidAddr_<4, tvm::runtime::CUDAWrappedFunc>(tvm::runtime::CUDAWrappedFunc, std::vector<tvm::runtime::detail::ArgConvertCode, std::allocator<tvm::runtime::detail::ArgConvertCode> > const&)::{lambda(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*)#1}>::_M_invoke(std::_Any_data const&, tvm::runtime::TVMArgs&&, tvm::runtime::TVMRetValue*&&)+0x9e) [0x7fe7e6fae74e]
  [bt] (1) /usr/local/lib/python3.6/dist-packages/tvm/libtvm.so(tvm::runtime::CUDAWrappedFunc::operator()(tvm::runtime::TVMArgs, tvm::runtime::TVMRetValue*, void**) const+0x755) [0x7fe7e6fae645]
  [bt] (0) /usr/local/lib/python3.6/dist-packages/tvm/libtvm.so(+0x15f4505) [0x7fe7e6faa505]
  File "/workspace/tvm/src/runtime/cuda/cuda_module.cc", line 105
  File "/workspace/tvm/src/runtime/library_module.cc", line 78
CUDAError: Check failed: ret == 0 (-1 vs. 0) : cuModuleLoadData(&(module_[device_id]), data_.c_str()) failed with error: CUDA_ERROR_NOT_INITIALIZED

I tested this with image: docker pull nvidia/cuda:10.2-cudnn8-devel-ubuntu18.04

while if build tvm from source, it works without any issue…

Anyone know what could be the cause of this?

Update: I got the same error using the Dockerfile.package-cu102 provided.

The error can be produced using https://tvm.apache.org/docs/tutorials/get_started/relay_quick_start.html#sphx-glr-tutorials-get-started-relay-quick-start-py

#10

Thanks for this initiative and it is commendable towards reducing the burden for use of the Apache TVM project.

Could you link to the Apache policy here for other folks to read and see what other guidelines need to be investigated as I couldn’t find it easily enough ? It might also be worthwhile for the project to document these aspects as items to consider.

Would this be something that would also allow others to add their own binary packages to this - I can envisage a linkage against the Ethos-N compiler / driver library for providing users with use of the work in Arm easily.

regards Ramana

#11

Because the package is thirdparty (non-ASF). The main policy that need to referred to is the trademark policy if the product would like to be referred to as “Foo Powered by Apache TVM.” See http://www.apache.org/foundation/marks/faq/#poweredby

We can also move followup convos specific to tlcpack to https://github.com/tlc-pack/tlcpack/issues

#12

Hi @tqchen, is that discussion still ongoing? Is there any help needed, to be able to host tlcpack on PyPI?

#13

This could be the case. however we figured out a mechanism that auto update to github binary tag, and the matrix in https://tlcpack.ai/ so far gives quite clear instruction so i think we are probably fine

#14

I’m working on getting Linux Nightly built for pip. To do this, I needed to upgrade to manylinux2014. It wasn’t possible to easily build LLVM with manylinux2010 any longer. Additionally, the nightly will only work for builds from main rather than the latest stable release. I think this is enough motivation to start working towards the 0.8 release, as there are many new features that have landed.

The next steps are to write a workflow to build the Docker image and publish the resulting package.

2 Likes
#15

@hogepodge thanks for doing the work on enabling nightly packages.

I got a question about the names of the packages: why do we need to have different names, as in tlcpack and tlcpack-nightly being published as different things?

cc @mjs @manupa-arm @tqchen

1 Like
#16

The main reason is that they corresponds to different hashtags, the nightly always points to the latest, while the non nightly points to a fixed tag(likely will switch to only stable version after v0.8)

#17

Hi @tqchen,

Im not sure whether that requires different namespaces for the packages.

Why cant we use something as follows :

  • Released versions: e.g.: tlcpack-0.8 tlcpack-0.8.1 tlcpack-0.9

  • Pre-release versions instead of tlcpack_nightly: tlcpack-0.10.devXXX

#18

The main thing is to make sure user expects exactly what to get. The versioning always follows the github version now. e.g. when you do

pip install tlcpack-nightly -f https://tlcpack.ai/wheels

You know that it is going to give you the latest nightly developer build. Mixing the namespace would require more specialized commands during installation. For users who do not want nightly build, it also helps because they can choose to use the one without nightly suffix and as a result not having to see the nightly builds.

The separation also makes the management easier, since nightly and stable are managed separated, we have script to cleanup stale nightly build to save the overall space.

#19

@manupa-arm i think the pip PEP has some more on this: PEP 440 -- Version Identification and Dependency Specification | Python.org

in particular the note at the bottom means we should probably keep only a small number of nightlies in the release package. it might be possible to keep more on our tlcpack pip index, though.

#20

Hi all, I wanted to touch base on this packaging/versioning topic.

I think it would be beneficial for our users to get the packages directly from pypi rather than hosting them somewhere else. It would be great so that people can set tlcpack as dependencies on their workflows.

Additionally, I think it would be great if we could keep a history of nightly packages so that users’ environments are reproducible. At the moment, if somebody notices an issue with a package, it is very hard to go an rebuild that package so that the issue can be reproduced.

With regards to pypi repository, I noticed we had a interaction with pypa in past: https://github.com/pypa/pypi-support/issues/594.

As we have ongoing discussions to increase the frequency of releases (see Release Planning - Reviewing our Tracking Issues), for the benefit of our users, I think it would be great to improve the tlcpack tooling so that we can streamline publishing on pypi.

So, I think there are a few actions here:

  1. Re-engage with pypa, in order to get adequate quota to host the packages
  2. Create the necessary community-owned credentials so that we can publish packages on pypi
  3. Improve tooling on tlcpack so that it is able to publish packages on pypi. I can start this one, but probably will need from others to fix issues and maintain this. I’ll raise the appropriate tickets on https://github.com/tlc-pack/tlcpack
  4. Amend TVM release process so that we guarantee that tlcpack gets some attention as well?

cc @tqchen @haichen @areusch @Mousius @ramana-arm for comments.

1 Like
#21

@leandron I support all of that. Do you have cycles to re-engage with pypa? It seems like we would do this for just the tlcpack package and not tlcpack-nightly?

I imagine our package retention policy might be somewhat informed by whatever quota we’re allowed. On the -nightly server, we can likely find quota if needed, but probably want to retain only e.g. a week or two.

Could you say more on this one?

#22

It relates to making sure that at some point in the release process we generate the tlcpack, including all supported configuration, and upload it to pypi.

In practice, it means amending the docs at (Apache TVM Release Process — tvm 0.9.dev114+g278173c18 documentation) to mention at least to create a ticket on tlcpack so that packages are created and uploaded properly, given the subtle distinction of TVM being an Apache project and tlcpack a community initiative.

I have some experience and ideas on how to amend tlcpack to make it able to upload to pypi, given we get credentials e quota for our packages, would you (@areusch or @tqchen or @haichen) mind to deal with points 1 and 2 on my list above, while I prototype the changes on tcpack repo?

#23

@leandron I could work on this in Jan.

#24

So, updating this thread again, we’re currently improving tlcpack so that is able to upload packages into PyPI.

During the investigation for these improvements, we realised we own the project name apache-tvm in PyPI, and as a way to make it simpler for our users to get hold of a TVM package, we’ll be using that namespace. The intended final result is that an user can simply pip install apache-tvm to get the latest CPU-capable release version of TVM.

Other packages will be available for GPU enabled builds, similar to what mxnet does: mxnet · PyPI. We can’t promise any time scales for this to be delivered. In case that is important for you, we invite you to get involved.

Current status:

  • We sorted out the required credentials for apache-tvm on PyPI so that committers who want to contribute with packaging work are able to (thanks @areusch and @driazati)
  • We have an ongoing PR that adds functionality to tlcpack GitHub Actions to upload a package to PyPI - https://github.com/tlc-pack/tlcpack/pull/102 (CPU only)

Next steps:

  • We welcome more contributions and people interested in helping with packaging. This is the only way to speed-up this process
  • Expand this work for GPU enabled packages
  • Collaborate with other initiatives in the TVM project so that we can link this works with a more frequent release process, as well as making the package available very soon after the release is declared in TVM.
1 Like
#25

We discussed this at the TVM Community Meeting this morning. Here are some notes:

  • How are we planning to mark versions?
    • PEP 440 discusses the different ways to denote a dev/release version. If we have a package named .devN, this signifies that it’s not a release version.
    • tlcpack infra already takes this into account.
    • After doing this, it will be possible to install nightly packages using pip.
    • The storage aspect is unclear, don’t know whether nightly releases are too often.
    • last-successful tag marks the latest version of tvm in repo.
  • Mehrdad: Are we going to have better dependency tracking in pyproject.toml as part of this effort?
    • That’s in parallel with this effort.
    • It would be nice to
  • Are we able to measure and track downloads to PyPI packages?
    • One of the side effects of pushing new packages nightly is it would reset the download counters.
    • David: pypistats.org reports stats based on project name. Also libraries.io
    • One challenge with the current scheme is that we both upload new packages and delete older packages. This makes it hard to track, and besides that makes it hard to reproduce issues (i.e. if the package is gone).
  • David: are we going to do any testing on the binaries we build? Doesn’t need to block these initial efforts. Would be nice to have at least some core TVM tests that run. Does this make sense?
    • Leandro: Not in place now, but is sort of vital.
  • David: Are we going to do a back-release of 0.8? Or just wait til 0.9.
    • Leandro: suggest to keep producing dev packages for now, and then do 0.9 release.
    • For the 0.9 point release, we’ll need to consider
  • Andrew: Are people going to find the package useful?
    • Gustavo: I just build from source
    • Sebastian: In some of our workflows it would help, in others we’d build from source.
    • Chris: Seems like for anyone who wants to try this out in e.g. PyTorch environments, this would be really useful. Packaging is a useful step in getting TVM into production, and compiling shouldn’t be the end goal.
    • This is probably the wrong group of folks to ask anyway :), we are all developers.
  • Leandro: packages also useful for us to track performance numbers
  • Gustavo: as users would use more of the packages, they will file issues and complain. Very helpful for us to determine the config.cmake and tag used to build that package. How can we promptly get this info based on a bug report?
    • Leandro: The version tag doesn’t have a place for the git hash. We could instead start tagging the package version in the repo.
    • David: Right now we have the point release listed in the python version, but we could have a lot more included.
    • Gustavo: And for the config.cmake, how can we find that?
      • Ideally we could check in the configs to TVM.
  • Are we going to test the packages at all?
    • Leandro: concrete example is when we needed to get standalone_crt in tvmc (extends to all data dependencies)
  • Next steps: progress the PR and check back in after about a month’s time here.
    • Would be great to start publishing CUDA packages. Help wanted here in collaborating with someone who has experience with CUDA. cc @junrushao if you could suggest someone.
2 Likes