Update README.md

[AnitaLiu98]

Fix the issue where solutions generated after executing the bash script are not being saved.

[loubnabnl]

The generations are saved by default in the generation_only mode

bigcode-evaluation-harness/main.py

Line 389 in e54f33d

evaluator.save_json_files(

[Vipitis]

I am also experiencing this. If you set --generation_only and --save_generations_path "./gens.json" but not --save_generations It does not write to file. Several examples and the README should be adjusted, else work might be lost.

problematic behaviour is this line:

bigcode-evaluation-harness/bigcode_eval/evaluator.py

Line 117 in e54f33d

if self.args.save_generations:

seems like an easy fix would be setting

self.args.save_generations = True if self.args.generations_only else self.args.save_generations

Or

if self.args.save_generations or self.args.generation_only:

[loubnabnl]

Ah I missed that, feel free to open a PR that adds if self.args.save_generations or self.args.generation_only: @Vipitis . Thanks!

[Vipitis]

Ah I missed that, feel free to open a PR that adds if self.args.save_generations or self.args.generation_only: @Vipitis . Thanks!

Will do, but won't be able to submit it until next week.