Can I try an example? Again, this is all motivated by working in the device planning code so my pov may be skewed.
Let’s say we’ve implemented the above and we have three target labels and their associated definitions:
cpu: llvm ...
gpu1: ... device_id=0
gpu2: ... device_id=1
I’m assuming a two gpu system without memory sharing. Perhaps they even have exactly the same microarchitecture and compiler options, hence I’ve include explicit device_id in the label definition. (I’d allow labels to ‘inherit’ from other labels so user’s could build up more specialized targets from prototypes, ultimately ending in a hard-coded target kind – there’s a lot of text in the above so forgive me if that’s the plan anyway).
I’m assuming these target labels can be used directly in ‘on_device’ annotations.
on_device(..., attrs.device="gpu2")
I assume the notion of ‘host’ is the same as the notion of ‘default device’. Ie it is as if the entire user model was wrapped in:
on_device(<model>, attrs.device="cpu")
I assume the user, or an intermediate transformation, may include additional annotations:
x = on_device(add(a, b), attrs.device="gpu1");
y = on_device(add(c, d), attrs.device="gpu2");
add(x, y)
The device – or target – planning is just flowing target constraints through the code. Two targets are compatible if they have exactly the same fields. device_copy is needed at any transition between incompatible targets.
Is that what we want?