kb @keighbee.bsky.social · 05/12/2024Preliminary data shows the Apple Neural Engine uses ~94% less energy than the CPU and ~75% less than the GPU 🤯 On the On-Device team at Hugging Face, we've been profiling energy usage for CoreML models. Here’s some data I collected: 241
greylint.bsky.social @greylint.bsky.social · 11/12/2024How can I get my torch models running on this? Thanks. 100
kb @keighbee.bsky.social · 05/12/2024To put it in perspective: Llama3.2-1B uses ~280 GFLOPS per 20 tokens. My laptop (~2kg) running the model would be the energy equivalent of: - CPU (6 J): dropping it from 1 foot (31 cm) - GPU (1.4 J): dropping it from 3 inches (7 cm) - ANE (0.3 J): dropping it by just half an inch (1.5 cm)! 120
