The winner of a fastest-code challenge is determined by the runtime performance of the submissions. For fairness, all submissions should be benchmarked on the same machine, which usually means all submissions have to be tested by the host of the challenge. Alternatively, the submission can be compared to a reference program. For scoring by asymptotic time complexity, use [fastest-algorithm] instead.
This tag indicates that the program with the fastest runtime speed is the winner of the challenge. There are several ways to determine each submission's score:
- Time taken to process a fixed set of benchmark inputs (or a single large input).
- Largest input successfully processed within a fixed time limit.
- Largest valid output produced within a fixed time limit (e.g. for number searches).
In any case, it is important that time is measured objectively, fairly and reproducibly. This usually means that the host of the challenge has to compile and run all submissions and test them himself, in order to run all programs on the same machine.
In some cases, there may be other options, like providing a reference implementation and taking the ratio of each submission's score to the reference implementation's score. This however does require trust in the participants that their scores have been obtained honestly.
