The Leela Piece Odds Challenge: What does it take you to win against Leela?

Finally, LeelaPieceOdds is ready, awaiting piece odds challenges from you at Lichess.

Summary of Leela giving piece odds so far

When we initially tested configurations for Leela giving piece odds with regular Leela nets using Contempt, quality of play deteriorated quickly with higher material value and more missing pieces. This was especially true when missing bishops, as the network seemed to hallucinate them back into existence, neglecting the arising pawn weaknesses. Thankfully, some nets performed well enough in selected positions, so we added LeelaRookOdds, LeelaQueenOdds and LeelaQueenForKnight to LeelaKnightOdds, forming the Leela Piece Odds family on Lichess.

Of these four, the two favourites have been the hardest (knight odds, at or above strong GM level) and the most accessible challenge (queen odds, performing at 2000-2700 rating, depending on the time control), with a total game time of 60 days of knight odds and 100 days of queen odds within less than a year. While the other two combined add 40 more days, they aren’t doing an ideal job at bridging the big gap between knight odds and queen odds, in particular due to the 400+ Elo gap between queen odds and queen for knight odds and the unfamiliarity with openings being a knight down as a human player well versed in theory. Furthermore, for a significant share of chess enthusiasts, especially at faster time controls even queen odds isn’t enough to overcome the raw strength difference to a top engine like Leela.

The missing puzzle piece: A dedicated piece odds network

Already when comparing the original configurations for Leela piece odds bots with the knight odds configuration, it was evident that the optimal Contempt settings for higher odds didn’t differ much from the knight odds configuration, suggesting that the primary goal is to allow reasonable opening play and avoiding forced piece exchanges, irrespective of the total odds.

Thanks to the new dedicated odds network, we don’t have to rely on Contempt anymore in order to capture potential difficulties in a position. This reduces the amount of nodes required to select reasonable moves (allowing more games to be played in parallel on a single RTX 4090), and more importantly also improves the quality of higher odds play, especially with multiple missing pieces. With this, playing almost arbitrary piece odds positions becomes viable, allowing a fine-grained division of the gap between knight odds and queen odds, plus an extension beyond queen odds to include lower rated players as well.

Since the odds network is trained on games where white is the stronger player, in order to play piece+move odds where black is the stronger player starting one or more pieces down, a special Lc0 binary is required, swapping colors when evaluating the NN with the SwapColors option.

Selection and difficulty of piece odds starting positions

Historically, in order to establish a more fine-grained sequence of increasing odds, removing pawns and allowing multiple moves in a row were used as well. This however comes at the downside of potentially affecting pawn structures and openings, and potentially accelerates Leela’s piece development, which effectively reduces the handicap. We therefore decided to only remove full pieces but no pawns, allowing players to experiment with their preferred setups.

Two more restrictions are made: First, if a rook is removed, it’s always the queen side rook in order to maintain king side castling rights. Second, removing one of the two bishops leads to an unrealistic color imbalance regarding the pawn structure far beyond the opening phase. We therefore only include odds positions where both bishops are either present or removed.

To assess the difficulty of the different challenges, the following approximate material values can be used:

Queen: 9.0 Rook: 4.6 Bishop pair: 7.0 Knight: 3.0

where 1 point in material equates to roughly 70 Elo performance. Furthermore, the difference between playing as white and as black is bigger in odds chess than in regular chess, approximately 100 Elo.

Challenge LeelaPieceOdds on Lichess

LeelaPieceOdds accepts the following starting positions.

If you accidentally select the wrong color (or try to play from an unsupported position), there will be a generic error message that the bot currently isn’t accepting challenges. Similarly, the Lichess rematch feature automatically switches colors, thus rematch offers will always be declined.

Piece odds (play as black)

N (b1) N (g1) R Qn NN BB RN Q RR BBN (b1) BBN (g1) RNN RBB QN (b1) QN (g1) BBNN QR QNN QBB QRN QRR

Bonus (won’t count towards the ladder): BN BN

Piece and move odds (play as white)

n (b8) n (g8) r qN nn bb rn q rr bbn (b8) bbn (g8) rnn rbb qn (b8) qn (g8) bbnn qr qnn qbb qrn qrr

Bonus (won’t count towards the ladder): bn bn

Future plans: Leela Piece Odds leaderboards

In order to highlight the community aspect of challenging Leela to piece odds games, we’re planning to add a leaderboard for games against LeelaPieceOdds, tracking and comparing the individual performances at different time controls and odds, and thus hopefully encouraging players to try setups both above and below their rating. In order to get more consistent data, only time controls 1’+1", 3’+1", 5’+3" and 15’+10" will count towards the ladder, and the bot might stop accepting other time controls automatically at some point.

A preliminary version of the leaderboard can already be accessed here for games against LeelaQueenOdds, showing the estimated individual performance at queen odds, corrected for the time control and color, recalibrated to 5’+3".

Acknowledgements

Making top engines accessible to the chess community in such a unique way is only possible thanks to the support of a multitude of people. This includes the lichess team and patrons for providing the online platform, the lichess devs and contributors for maintaining and developing specialized tools like lichess-bot and direct challenge links, and GM Larry Kaufman for his continuous contributions judging and improving the quality and performance of Leela’s piece odds play. Credits for training the specialized knight resp. queen odds nets go to marcus98 and Noah.