Additionally, bare metal devices typically have very limited memory (measured in KB).
8G or 16G of memory size are very common now. I am not quite sure what “bare metal devices” mean here.
I think the most useful definition here might be:
Anything that does not run a full operating systems (RTOSes do not fall under that definition). The key difference being that you can’t rely on the OS for memory allocation, device abstraction, management, swapping, threading, File IO, etc. Instead, the entire inference needs to be self-contained as either C code are an ASM binary.
This is typically seen in embedded devices (think ARM Cortex-M MCUs) which do have somewhere in the low MB (at best) of persistent and non-persistent memory.