how to reproduce the benchmark score?

by lincharliesun - opened 17 days ago

17 days ago

Just as mentioned in the README,

"We run the benchmarks up to 16 times and average the scores to be more accurate."

I'm curious. For taking the average of 16 times, is it sampling 16 times with the same seed or sampling once for each of 16 seeds?
Will there be a big difference between these two methods?

thanks

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

Your need to confirm your account before you can post a new comment.

· Sign up or log in to comment