Recently, the transformer architecture has dominated domains as diverse as vision and natural language processing. Over the past two years, the Lc0 team has been trying to answer the following question:
What chess-specific enhancements can be made to the transformer architecture?
To explore the performance of Lc0 networks relative to DeepMind’s state-of-the-art transformer networks, we embarked on a comparative analysis, inspired by the methodologies detailed in DeepMind’s latest publication. Our objective was to closely align our testing approach for Lc0 networks with the evaluation framework applied by DeepMind, allowing for a direct comparison of results.
Since its first games almost 3 months ago, LeelaKnightOdds has played over 1800 matches against a variety of opponents at a multitude of time controls.
In the recent blog post presenting the new WDL contempt feature added in the Lc0 v0.30 release, we shared our plans to add a lichess bot for piece odds games. While allowing arbitrary piece odds poses several challenges, which aren’t resolved yet, we are proud to announce that we made a big first step towards that goal with LeelaKnightOdds, now accepting your challenge on lichess.
The imminent v0.30.0 Lc0 release has two main features, attention body net support and WDL rescale/contempt. This blog post is about the latter, which is continuing our past efforts on providing more realistic WDL predictions with Lc0.
2022 has been a great year for Leela. A lot of new contributors appeared and made significant improvements; overall, Leela has become considerably stronger and more interesting. This year has brought huge changes to how the search was conducted, network architecture, available backends and more.
It’s been a while since we released a new version of Lc0, but we finally put out v0.29.0 a few weeks ago. In this post, we’ll talk about what’s new in this release and why it took so long to come out.
In the Leela Chess project, we generate a huge amount of data. We use them to generate the network files to use with Lc0 for further data generation, but also with other chess engines, like Ceres. The same data are often used by individual project contributors to generate additional network files using the “supervised learning” approach.
It all started a couple of months ago. First the Stockfish 13 release announcement and shortly later the Lc0 v0.27.0 one contained identical language that both “teams will join forces to demonstrate our commitment to open source chess engines and training tools, and open data.” While the intention is still there and we stand behind this statement, we haven’t yet managed to make something more formal in this direction.
With this in mind, when I was looking for an April Fools’ joke for this year the idea to use LeelaChessZero training data to train a NNUE net seemed very enticing.
Unless you were living under a rock during the last year, you have probably heard of the revolution that has been happening in computer chess. That is assuming you are interested in computer chess, but if you are not then why are you reading this? We are talking about Efficiently Updatable Neural Networks (referred to as NNUE, giving new meaning to backronyms) allegedly discovered by Japanese monks on sacred FORTRAN punched cards. The introduction of NNUE to the Alpha-Beta search of Stockfish resulted in impressive gains, despite initial bugs and ridicule. Since then the dominoes have been falling one after the other and now almost all the top chess engines have a NNUE implementation. Obviously Lc0 couldn’t be far behind, so we proudly present LcFiSh, the latest incarnation of Lc0 with NNUE technology.
There has been a long time without posts in our blog, and finally there is a good occasion to revive it.
One of the most active LCZero contributors dje, was not very active in 2020. There was a good reason for it though — he was writing Ceres, a new chess engine that uses LCZero networks!
Yesterday, December 31th 2020, it was released under the GPLv3.
Lc0 v0.25 has been released.
(and it’s actually already at v0.25.1 due to hot bugfixes after the initial release).
thanks to @dtracers and @ildolphino for correction and rewriting.
Leela won the SuFi, 52.5 to 47.5 (+5 wins). This was a difficult match that showed the qualities and the weaknesses of the two engines. Leela, with network SV-t60-3010, is by most people considered as a positional engine who likes to grind down the opponent instead of going for the quickest mate, but of course she is also a tremendous tactical player. Still she has tactical blunders and sometimes has difficulties with correctly estimating the opponent’s queen play, especially in open positions. Sometimes she does not succeed in converting a winning position because of falling prey to a perpetual check (note: the developers are working hard on this issue and a few enhancements are in the pipeline). StockFish, with 176 threads, is a fast and impressive tactician with a very solid play. But, even non-grandmaster players permit themselves to call its positional play dubious at times. It looks like a very clever player, it calculates good combinations but sometimes does not know how to coordinate pieces with pawns. In other words StockFish attempts to solve chess by brute force calculation, very clever search heuristics and a fast evaluation function. Sometimes it even helps Leela in some positions sacking pawns in a desperate way while Leela had a hard time finding the win!
Traditionally, computer chess engines evaluated position in terms of pawns. This is a convenient choice, as material is a foundation of position evaluation, which is true also among humans. Situations where the importance of positional aspect strongly outweights material are often thought as beautiful exceptions rather than the rule.
It is however not the most logical way to score a position. Using pawns sounds arbitrary, and it’s also not linear and the same step has different meanings: while going from 0 to 1 pawn advantage keeps the position quite holdable, going from 1 to 2 usually makes it mostly lost. Going from 10 to 11 makes no difference at all.
One of the commonly expressed complaints about lc0 is that there are too many parameters. While this is undoubtedly true, this gives us unparalleled flexibility to tune the engine’s behavior to reach the point where “when you see what Leela suggests, then you say - it makes sense” as Magnus Carlsen recently said. While there are efforts underway to simplify the user-facing configuration options, we decided that this is a perfect opportunity to reveal the underlying formulas used to derive the default values for most parameters:
IMPORTANT UPDATE: v0.24.0 had a bug when either
--draw-score-* options were active. If you use either of those options, please
upgrade to v0.24.1.
It’s been a while since we had a blog post, and it’s been a while since we had a release of Lc0. And today is happening:
Lc0 v0.24 has been released, and you can download it here.
Update: we have a bugfix release v0.23.1, download here. It contains no chess-play-related changes, the bug was in training game generation code.
We have a new release today!
The era of test40 has finished, that training of test40 has been stopped.
Does the world need another Chess GUI? Some people say no, but they don’t have very many Chess GUIs, do they?
I’m the author of Nibbler, the first Chess GUI designed specifically for Lc0, and inspired by the excellent Lizzie Go interface. Although Nibbler is a work in progress, it’s very usable, and has received a lot of favourable comment.
I’ve just decided that maybe it’s a time for a post. Describing what’s going on and what are the plans, stuff like that!
There are some plans for the Lc0 engine itself, but that is a topic for another post. At least we hope to make releases more regular again (v0.21.0-rc1 appeared on February 16th, and we are not even v0.22 yet).
After quite a long delay we are releasing an update to Lc0, new version is v0.21.2.
2019-05-26 UPDATE: Temporary server is up again.
2019-05-25 UPDATE2: Turns out cloud instance was preemtable, so it was preempted, no training again for now.
2019-05-25 UPDATE: We’ve been donated a temporary AWS cloud instance for the time of the main server downtime! Training is back up, but only for test53, test40 is still paused.
2019-05-23 UPDATE: PSU seems to be in order, replugging all cables didn’t help. The person who has access to the server has to travel again, no news expected until next week. :-(
2019-05-20 UPDATE: The server doesn’t react to a power button, so it’s not something trivial. Will take a more thorough look tomorrow.
2019-05-17 UPDATE: The person who has access to the server is currently travelling, until Sunday. That means that updated information about the server is expected in the beginning of the next week.
Today, May 10, 2019, Leela is playing in the first game of the high profile TCEC Season 15 Superfinal. Leela qualified after she cleared Division P undefeated, winning at least once head-to-head against every opponent except Houdini. Her opponent in the superfinal is Stockfish, who lost once to Leela head-to-head in Division P, but finished with a higher score by winning more games than Leela against the other Division P competitors.
To break the silence of lack of posts in this blog, let me write about Lc0
backends configuration, as it has been totally undocumented so far.
(For other options it’s possible to get rough idea of their meaning by running lc0 –help).
Announcing LcZ, the world's first neural net based chess engine and interactive fiction amalgamation.
After the huge success of the xyzzy extension to the UCI protocol introduced in Lc0, it was decided to take it one step further and introduce z-machine compatibility to the chess engine, giving birth to LcZ.
This a quick note for people who use CuteChess and just upgraded to v0.21.1 from earlier versions.
Meaning of FPU-related parameters are changed in v0.21.1 , along with default values of those parameters.
The lczero.org server will be not available for ~24 hours starting at April 4th 20:00 UTC.
Training game generation will also not be possible during the downtime.
We recently did the 2nd LR drop for T40, and usually this is when the net approaches it’s strongest point (gains after the final LR drop are very small). External Elo tests show T40 is close to T30, and may have already passed. We will know more in the following few weeks. But coincidentally we found two major issues at the same time as the LR drop:
Lc0 did well in it’s first TCEC Superfinal appearance, but fell just short of winning the match, losing 49.5 to 50.5. Here are some overviews of the Superfinal:
As per the usual expression of BOOM of TCEC chat when an engine finds something good, Leela right now is making a great surprise in her first appearance in a TCEC superfinal by leading after 64 games with 2 points more. A 33-31 score in favor of Leela.
TCEC superfinal of 14th Season is currently being played as
Stockfish and Leela battle for the TCEC Season 14 Champion title.
Till now Leela surprises Stockfish and after 33 games the result is a perfect tie with 16.5-16.5 points.
Yet 67 more games will be played so everything can happen.
Leela has won the TCEC CUP-2! After many very difficult battles and games
against the top Chess engines, Leela eventually managed to win the tournament.
In the final Leela managed to beat Houdini at the very final game(before tiebreaks start) with a spectacular win.
As always, consult the [glossary](https://github.com/LeelaChessZero/lc0/wiki/Technical-Explanation- of-Leela-Chess-Zero) if needed. Ordo rating The standard way to to put a number on anyone’s chess skills is to determine the Elo rating. There is however an alternative called [Ordo rating](https://www.
[!(../../images/2019-01-16-leela-promotes-to-superfinal-of-tcec- jkhkjh9879879999.png) Leela just made it on the superfinal of the TCEC tournament! There, she will face Stockfish on a 100-games match for the title of TCEC champion. As always with Leela it was a dramatic promotion at the last moment, at the last game.
Quick recap Remember to consult the [glossary](https://github.com/LeelaChessZero/lc0/wiki/Technical-Explanation- of-Leela-Chess-Zero) if you find some terms confusing. With Test20 being suspended on November 16th, we started December with Test30 as the only game in town.
Not everyone knows, but recently there was a match between Lc0 and GM Daniel Naroditsky on Lichess. For those tho missed, here is recording of this stream on Twitch and [Lichess Blog entry](https://lichess.
Lichess.org will host a match between the mighty **Stockfish 10 **and Leela. It will be a 6 games match with time control of 5'+2” with ChessNetwork commentary. Games will be played on 15th December at 17:00 UTC.
Leela’s big journey to try to go to premier division of TCEC, has started! TCEC season 14 is running for the last couple of weeks and Leela has participated in 3rd division of it, finishing in the top position easily and now participates in the 2nd division trying to promote to 1st division.
As everyone has already heard, DeepMind has published a detailed paper on AlphaZero! The announcement can be found [here](https://deepmind.com/blog/alphazero- shedding-new-light-grand-games-chess-shogi-and-go/). Scroll down the announcement to get links to the full paper text as well as supplementary materials (including PGNs of games and training pseudocode).
v0.19.0 is finally out of “release candidate” status, and now is fully released! It has been quite a long bugfixing run with 5 release candidates, but now all known issues seem to be resolved.
The play.lczero.org web site where everyone could quickly play Lc0 online is currently down. But even while it doesn’t work, there are some options to play Leela online. The easiest way is to play on lichess.
The release candidate of a new Leela version has been released: (v0.19.0-rc1) Upd: we are releasing v0.19.0-rc2 immediately as due to mistake in the release procedure rc1 reported its version as v0.
If you are new to Leela (Lc0) Chess and have begun contributing games either using Google Cloud or some other online service or your own home computer, you may be wondering where all those games go and how training of Leela happens.
Leela in a classic drama style, promoted in TCEC Cup Semi-Finals and it will face Stockfish today! While in CCCC blitz tournament she is still at 3rd place ahead of Komodo, Ethereal and Fire and behind Stockfish and Houdini.
CCCC blitz tournament is running and till now Leela is having a good performance being steadily on the top 3. Conditions for the tournament are: • 33 engines play a 4x Round Robin tournament with each engine that will play each other 4 times(2 with black and 2 with white) in a total of 128 games per engine, with no opening books or predefined positions used.
Chess is a game where there are 3 distinct results. White wins or black wins or it is a draw and nobody wins. Draw can achieved in many ways in Chess.
_ Article by Cyanogenoid, member of Leela Chess Zero development team ._ Recently, Oracle investigated training the value head against not the game outcome z, but against the accumulated value q for a position that is obtained after exploring some number of nodes with UCT [[Lessons From AlphaZero: Improving the Training Target.
Leela after 6 consecutive draws in the series of games against Laser(division 1 engine) she won last 2 games so she promoted to next round where she will face Ethereal(premier division engine) that beat Rodent(division 4 engine) easily with 5-0 and promoted too.
When first Chess engines were born were very weak. First real Chess program(even though in 1912 there was a King Rook versus King solver and in 1951 Turing wrote only in paper a Chess program) was written in 1951 by Alan Turing’s colleague, Dietrich Prinz, but it was not able to play a whole game but could only solve small “mate in 2” problems.
TCEC CUP running. Leela in danger of elimination from the 1st round against Laser! UPDATE: Leela won!
TCEC Cup continues and we already know 6 engines that have promoted to next round. Leela right now has a big fight with Laser and after 4 games it is on 2-2 with 4 draws.
The TCEC Cup has started and 2 match pairs will be played every day. We remind that 32 engines will compete in a Knockout elimination tournament, and in each pair match, the best out of 8 games will promote to next round.
Leela won 16-14 against Komodo the 30 games match to determine 3rd position in CCCC1, while on the variants tournament she won a game against Stockfish while being handicapped having a Pawn less!
It turned out that the version v0.18.0, that was released a few days ago, had a bug that caused it to miss tablebase wins sometimes. Because of that, v0.18.1 is released.
Deepmind’s latest paper release ( Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm ) on 5 December 2017, presented a method of reinforcement learning for convolutional neural networks that its result was fantastic as they created 3 engines, one for GO, one for Chess and one for Shogi that were beating easily the best available engines of the time.
TCEC Cup is about to start in a few days and Chessdom.com just announced the complete pairings up to the final. Leela is on the Stockfish side and if she will promote from the first match against Laser she will probably face Ethereal in round of 16, so it will be a tough match already.
v0.18.0 is out of “release candidate” status, and now is fully released! Can be downloaded here. It has no changes relative to RC2. For the list of differences relative to v0.
The TCEC(Top Chess Engine Championship) tournament is currently running with engines playing in the premier division and this division will finish in a few days. But this time, after premier division end, there would not be an immediate starting of the superfinal between the top 2 placed engines in premier, but a new tournament is going to happen that Chessdom has announced, the TCEC CUP , with all engines of TCEC but in a different format, that of knockout games, one that brings in mind the traditional Cup competitions of football.
Chess.com has [announced](https://www.chess.com/news/view/stockfish-houdini- to-battle-for-computer-chess-championship-komodo-vs-lc0-for-3rd) that after the CCCC superfinal between Stockfish and Houdini(Stockfish is ahead 27.5-20.5 till now) will finish, then 30 games of Komodo versus Leela will be played to determine the 3rd place.
The “Release Candidate 2” for the Lc0 version v0.18 has been published! Available to download here. Change: Fixed a severe race condition bug when only happened when –out-of-order-eval is on (which is the default value now).
Chess.com Computer Chess Championship stage 2 has finally ended. After a 280 games battle only the top 2 engines will continue to the superfinal. Stockfish and Houdini for a 200 games “fight”.
The release candidate of a new Leela version has been released: v0.18.0-rc1 We expect testing phase to last around 7-10 days, after which proper v0.18.0 will be released. Download and full changelog here.
Leela Chess Zero is a project started before some months inspired by Deepmind’s papers about AlphaGO Zero and AlphaZero, which is based on a new paradigm of Chess engines by not using traditional AlphaBeta search with handcrafted evaluation function but uses a variant of MCTS search called puct and for evaluation function it uses a self-taught neural network that learns by deep learning methods by playing against itself million times.
Stage 2 continues with Leela doing excellently being 3rd, half a point ahead of Komodo! ** Stockfish** continues its unbeaten run, being the only engine with no loss, stage 1 and 2 combined!
Stage 2 of CCCC has already started before 2 days. Not without problems initially with repeating restarts but at last now it’s running flawlessly. Stockfish continues to be invincible! As it hasn’t lost a single game in the first 46 games of stage 1 plus the 15th till now in stage 2.
1st stage is almost over, 1 round remains but the top 8 engines that will participate in stage 2 are known. Stockfish, Komodo, Houdini, Leela, Fire, Ethereal, Andscacs and Boot promote to next round.
While test20 runs, we are running test30 in parallel to test two ideas. First test30 uses a different method to initialize the first random net, and a slightly different LR (Learning Rate) strategy for the first few nets.
In CCCC there have been played some very nice games and moves till now. Can you find them? Play the move on the board and if it’s correct it will show the game continuation with a comment.
CCCC continues and Leela , along with Stockfish , Houdini , Komodo and Fire , already has secured a place in the 2nd stage. Ethereal after a strong finish, mainly winning in the last games, has almost secured the promotion.
First round CCCC approaches to its end and situation is clear by now. Houdini, Komodo and Stockfish will be in top 3 advancing to stage 2, with Stockfish and Komodo to fight for the 1st place in this 1st stage, thatdoesn’t mean much anyway, Fire and Leela would most probably advance to the next round also with some minor chances of not achieving it, and Shredder, Booot, Ethereal and Andscacs will fight for the other 3 places for stage 2.
When doing machine learning it helps to use a standardized dataset such that methods can be compared in an objective manner. For machine vision, one of the earliest standard datasets is MNIST, a set of handwritten characters that was also used in the (arguably) first deep learning paper.
It seems Leela is hiding something that is beyond us. How else can be explained that after Fizbo a 3325 CCRL Elo engine with endgame tablebases in a dead 4-men position lost to Leela, now Leela in another dead draw position made Xiphos, a 3200 CCRL Elo engines, to lose!
Leela had its first loss in the tournament after 27 games that went undefeated. It lost against Ethereal with black after she blundered in endgame. But before that, a real circus game happened in the Leela game of course again.
ROUND 23 Andscacs won Arasan in an important win for top8 place, while Ethereal and Booot drew their games. Top 3 continued with wins but Shredder drew. Leela took advantage of it and won against Laser so she is now alone in 4th place.
First half of the CCCC tournament has ended. It’s a 46 rounds tournament and so 23 rounds have been played. Time control is 15'+5” and engines will play all against all twice, one with black and one with white, while pondering is on and no opening books are used and 6 pieces tablebases are being used.
As it seems 6 engines will fight hard for 5 places that lead to next stage provided that Stockfish, Houdini and Komodo will have no problems to get the 3 top places.
CCCC continues with Houdini on the lead a full point ahead of the 2nd Stockfish. Houdini till now seems unstoppable and is killing every other engine except for the top ones.
Chess.com Computer World Championship continues with engines battling for the first 8 positions that would bring them to next stage. Round 17 has been completed out of 46 that will be played.
** **After another 3 rounds have completed and we reached round 16 of 46 that would be played, Houdini continues its comfortable lead being 1 full point ahead of the second.
As many of you have noticed, Leela’s thinking time allocation seems suboptimal for during the CCCC games. It almost doesn’t spend any time in the opening, and spends a lot in the endgame.
ROUND 11 This round saw Leela drawing rather not that comfortably a strong engine and currently in the top 8 engines(the top 8 of 24 engines are promoted to next round) Andscacs.
Some statistics about Chess.com Computer Chess Championship till now. Game length Frequency 1-0 =-= 0-1 1-0 =-= 0-1 All games 78 94 76 33.8% 46.2% 19.8% ** Shortest wins (White)**
](https://www.chess.com/computer-chess-championship) ROUND 9 It was the round of draws, with only 4 games to be decided and 8 draws. All top 6 engines drew their games. Ethereal had a pleasant game as it crashed the aging Crafty while Shredder couldn’t take anything out of the opening it had and Laser had no problems so game ended a draw.
The Chess.com computer Chess championship continued with most top engines winning their games so little things changed in the top positions. Leela had a good performance winning 2 games and drawing the 3rd against the mighty Stockfish!
Traditional Chess engines with classic Alpha-Beta search, sometimes have huge problems with some positions that require deep planning and understanding of the position. This is because they try to search all possible moves in a position and then their replies and then all the replies of the replies, etc. They discard most moves of course with clever algorithms that prune almost all moves but a few that they focus on. With a cost of course to discard a move, not good looking at first sight, but that after many moves ahead it proves to be a very good one. Traditional engines also have a handcrafted evaluation function. That means in order to judge a position they have to rely on human rules for it, e.g bonus for Rook on open file, bonus if a Pawn is on 6th rank etc, etc.
CCCC rounds 3 and 4 and 5 were finished. Houdini is on the lead with 5.0/5 while Leela is doing well with 4.0/5. In round 3 Leela with black against Fizbo, reached a material imbalanced position but with her King exposed she couldn’t avoid a perpetual check with a draw.
CCCC continues running and we are on round 3 currently. The level of play is
very high as it was expected and interesting games and positions arise.
Shredder is on the lead now with 3 points out of 3 games but other engines can reach that too(Komodo, Houdini).
It’s not completely relevant to Lc0, but many people who follow CCCC wonder how to disable sound.
Test20 is being run for a few hours already, and several people expressed a concern that this time progress is slower then it was in previous runs. Slower progress at start is expected (per network, not that much per time), for 3 reasons:
Chess.com Computer Chess Championship starts today.
24 engines will participate playing all against all twice, in a double Round
Robin tournament with 15 minutes for each player for the game plus 5 seconds
per move increment and pondering(thinking in opponent’s time) on. There will
be no opening books usage for the 1st round. Every engine will calculate all
the moves by itself.
Leela will play on four Tesla V100 GPUs while the other engines on 46 threads of a 2 x Intel Xeon Platinum 8168, 2.70 GHz that has 48 logical cores and 96 threads.
The hardware is very fast, the engines belong to the top ones so the level of
play will be amazing.
Every engine will play 46 games so there would be 46 rounds.
After all games are completed, the first 8 of the 24 engines will advance to round 2.
After the restart, the initial training of test20 (from random games) is not working as expected.
Some network properties are not going where we expected them to go (for example, it’s expected that MSE loss would suddenly drop, but it didn’t. Actually, it jumped up instead, can be followed here). Something is wrong with the training, and we are investigating.
As most of you are already aware, Leela will participate in the upcoming season of CCCC!
CCCC (chess.com computer chess competition) is a tournament, where top chess engines compete in a different set of formats, settings and time controls on a high-end hardware. Chess.com did conduct computer chess competitions in the past, but this time CCCC features a really good brand new shiny interface which will make watching it even more fun (and also this is the first time Leela participates there, that also adds fun :-P).
v0.17.0 is out of “release candidate” status, and now is fully released!
Can be downloaded here.
As it has been announced earlier, Leela has a partial endgame tablebase support now.
The support in v0.17.0 is partial only, only WDL tables are probed, but not DTZ. That means, that Leela is only able to query tablebase for positions immediately after captures and pawn moves, and for other positions it has to think by itself.
As you know, we are releasing Lc0 v0.17.0 to participate in the next CCCC (chess.com computer chess competition) season, which will be the first season with a new updated design.
The learning rate for the test10 training run has been lowered to 0.0002. Network id 11013 will be the first network trained with the new LR. This is the last time we lower it for test10 to squeeze some more Elo out of it.
The “Release Candidate 2” for the Lc0 version v0.17 has been published! Available to download here.
We had numerous issues in network encoding in the past, and now after pretty long pause we found yet another one! :) Turns out, that information about 50-move-no-capture-and-pawn-move-counter was located in wrong place in training data, so networks were trained without that information.
The release candidate of a new version of the Lc0 engine has been released. v0.17.0-rc1 We expect to have a stable v0.17.0 release in one week, so that we can use it for CCCC.
Welcome to the new Leela Chess Zero blog! The old blog is gone, and here it is, new and fresh. We will write new posts here. You can find old posts here: http://archive.
Two days before start of TCEC season 13, it was revealed that “DeusX engine” is just a set of NN weights trained by Albert Silver for the Lc0 engine. The LCZero team had a very short period to analyze the situation and react to it, especially given that members of LCZero core dev team were in different timezones or on vacation, and given that even after the announcement it was not completely clear what the “DeusX engine” exactly was and which tools were used to produce the weights.