This work is four years old (with development happening before that), so the Julia GPU capabilities probably weren't good enough at the time. If you wanted to do it today, that'd probably be the way to do it, but would need some benchmarking.
A lot of modern supercomputer use/have GPUs. But most GPUs had very bad fp64 compute capabilities, so they were not really used for anything requiring precision for a long time.
