Use evaluation when compiling modules

[josevalim]

This explores a different approach to execute module definitions that uses the evaluator (interpreted) rather than the compiler (compiled). I have tried this a long time ago but it was never faster, but I assume a combination of the JIT and different optimizations have made it viable.

Early experiments are very promising. This mode is 5x faster for Remote's codebase compared to 1.19 and 3x faster than main (which already has other improvements:

Version	Elixir Version	Elixir Commit	Total Time	vs Baseline
Baseline	1.19.4-otp-27	N/A	7:47.95 (467.95s)	-
Main	1.20.0-rc.1	426a429	4:57.91 (297.91s)	36.3% faster
This branch	1.20.0-rc.1	732e701	1:25.08 (85.08s)	81.8% faster

The next steps is to expose this as an option and do some additional testing to verify stacktraces are solid. Note this does not change the generated artefact in any way. Each function in the module is still compiled and optimized within the generated .beam file.

[josevalim]

I sent a PR for documentation purposes. I'm not planning to merge it for now.

[michallepicki]

It may be something very specific to the Remote codebase, or at least I am not seeing any significant improvements over main at compiling my work project, compile times are pretty much the same, maybe slightly faster test loading (9.8s vs 10.3s in mix test --exclude test output), maybe slightly slower project compilation (12.7s vs 11.8s time mix compile --force). Total compilation time including deps with parallelism 6 is also very close (~52s vs ~54s rm -rf _build && time mix test --exclude test).

[josevalim]

@michallepicki I could not measure such a large difference in Livebook as well. My measurements tell me that this is faster because it avoids the single core bottleneck of the code server, so the more modules and the more cores, the more likely this will yield positive results. The overall goal is to expose this as an option.

[exception-one]

Have you measured memory usage of this change?

This seems to significantly use more memory than the current behaviour. At least with the project I'm trying. When I try to compile the project inside an Ubuntu VM with 16 giga memory, the OOM killer always reaps the compilation. In htop I see memory usage of up to 21 GiB. Whereas the current behaviour barely uses 500 MiB of memory.

Out of memory: Killed process 889020 (beam.smp) total-vm:22154388kB, anon-rss:14060216kB, file-rss:2304kB, shmem-rss:1408kB, UID:1000 pgtables:27980kB oom_score_adj:0

Tested with latest OTP 27 and 28 release.