lichess.org
Donate

From Reliability Testing to Calibrated Learning.

Chess bot
Moving from reliability testing into training: focused on evidence, calibrated learning, improved decision-making, and building a stronger chess system.

1. Looking Back: The First 100+ Games

ApproximationBot has now reached an important milestone: over 100 completed games, with 112 games played at this transition point.
The first phase of development was about answering a basic question:
Could the system reliably participate in real online chess?
Through actual competition, the project discovered valuable lessons about:
- challenge handling
- game reliability
- move timing
- clock management
- recovery after unexpected events
The early challenges were not failures of the project. They were valuable data points. Each problem revealed something that could be measured, analyzed, and improved.
The first goal was survival: building a system capable of reliably participating in real games.
The next goal is improvement: learning from those games.
---

2. Reliability Improvements

Before meaningful learning can happen, the foundation has to be stable.
Several improvements were made to strengthen ApproximationBot's reliability:
- improved challenge handling and acceptance logic
- stronger state persistence
- better recovery behavior
- improved handling of interruptions and unexpected situations
- improved coordination between game operation, recovery, and background processes
Reliability is not a single feature. It is the foundation that allows everything else to improve.
A chess system cannot learn from games it cannot reliably complete.
---

3. Baby Evidence: Learning From Real Games

The next phase introduces a simple but important idea:
Baby Evidence.
Instead of assuming that changes make ApproximationBot stronger, the project focuses on collecting evidence that improvements actually occur.
The project can now focus on identifying:
- repeated mistakes
- opening patterns
- tactical opportunities
- clock-management decisions
- evaluation changes over time
The objective is not just to play more games.
The objective is to learn something meaningful from each game.
---

4. Calibrated Learning

ApproximationBot is entering a phase of controlled improvement.
The process is:
1. Observe
2. Measure
3. Adjust
4. Verify
The goal is not unlimited complexity.
The goal is making careful improvements supported by evidence.
A change should have a reason behind it, and improvements should be tested over time.
This approach treats chess development like an engineering process: build, measure, refine.
---

5. Time Management: The Chess Clock as a Resource

One of the major design ideas behind ApproximationBot is treating time as a resource.
The chess clock is not just a countdown. It represents a limited resource that must be managed carefully.
The objective is to avoid two problems:
- losing while significant time remains unused
- running out of time before the position is decided
The clock-management framework balances:
- chess quality
- survival
- calculation time
- uncertainty about game length
Every move represents both a chess decision and a resource decision.
The objective is not to spend the most time possible, but to spend time where it creates the greatest improvement in outcome.
---

6. Expanded Time Controls

ApproximationBot has expanded beyond its original testing environment and is continuing to evaluate performance across different time controls.
Different time controls provide different information:
- faster games test efficiency, reliability, and time management
- longer games test calculation and decision quality
- increment games test resource management
The goal is not simply to accept more formats.
The goal is understanding how different environments affect performance and improvement.
---

7. What Comes Next

The next stage is continued training and evidence collection.
The focus areas are:
- collecting more real game data
- analyzing performance patterns
- improving calibrated learning
- measuring operational reliability
- continuing to refine decision-making
Blog 4 will review approximately one month of training-phase results:
What improved?
What stayed the same?
What evidence was discovered?
ApproximationBot is still a work in progress.
The biggest change is that the project has moved from only building the system to learning from the system.
The next phase is not about claiming perfection. It is about collecting evidence, making careful improvements, and continuing the journey.
Training today.
Stronger tomorrow.