The major difference with LLVM is modularity. If you create a new language frontend, you get all LLVM's CPU support "for free." If you create a new processor backend, you get all LLVM's frontend languages "for free."
This has interesting consequences such as retargeting anything from the frontend to anything on the backend. I'd venture a wager that in the old mainframe days, the monolithic nature of a JIT would not have been friendly to a porting campaign.
The difference is that they had to port the entire JIT to the new processor. With LLVM, you just need the backend components that represent the CPU. Granted, from an app developer's perspective, this is not really relevant because they're targeting a VM; where the VM runs, their apps run.
This has interesting consequences such as retargeting anything from the frontend to anything on the backend. I'd venture a wager that in the old mainframe days, the monolithic nature of a JIT would not have been friendly to a porting campaign.