Voting Changes - Schulze
The ranking-projects page suffered from a couple of biases that have led to a complete revamp - to be used for the first time when ranking projects in Spínat later this month.
With the referendum looming large, it seems appropriate that Naglasúpan undergoes some voting reform of its own.
Existing Ranking Biases
Every reviewer delivered a full ranking of all projects, the projects were pre-filled, and in the same deterministic order for all reviewers.
This led to the possibility of someone who cared only about the top 3 leaving the others in the same order that they arrived in.
The ordering was alphabetical meaning that projects earlier in the alphabet were less likely to be hit by this tax, whilst later letters had an increased risk of getting low "default" scores.
Having looked through the results there's no evidence that this has affected any competitions, but the risk is there and so it's been changed.
Schulze Method
The Schulze** method is a "Single winner ranked-choice voting rule",
The idea behind Schulze's method is that if Alice defeats Bob, and Bob beats Charlie, then Alice "indirectly" defeats Charlie
How it works:
- Every ballot becomes a set of one to one comparisons: A project ranked by a reviewer counts as preferred over every project they placed below it, and every project left unranked.
- Those comparisons are added up per pair. A margin is calculated between pairs: Number of ballots preferring X over Y minus the number preferring Y over X.
- Projects are ordered by strongest chain of victories. X is placed above Y when the strongest chain running from X to Y is stronger than the strongest chain running back. A chain is only as strong as its weakest link: if X beats P by 5 and P beats Y by 2, the chain X -> P -> Y is worth 2.
- Top strongest project is selected the winner
All praise of the choice of ranking strategy goes to Samúel Hafsteinsson.
Unbiased Ranking
Users now see a pool of projects and choose which ones they want to rank - they do not have to rank them all.
The pool has a deterministic per-reviewer order meaning that there is no bias in the presentation of projects.
Tie-breaking
It is possible for Schulze to produce ties. When that happens, these are the tiebreakers that we now go through, in order:
- Least-bad worst defeat between the tied projects. For there to be a tie we must have a loop where each beat each other. Who lost the least within that group?
- Number of reviewers that ranked the project
- Best mean position by those that ranked it.
- Most first-place votes
If after all of these the projects still can't be separated, it's safe to say it's a joint victory.
The implementation can be seen in this PR: https://github.com/alexcouper/nglspn/pull/74