When would I approve a large pull request made by a magic genie?
Written by Grigory Rechistov in Uncategorized on 31.03.2026. Tags: testing, mutation testing, llm,
Preword
Recently I was asked to review a …
Posts Tagged With 'testing'
A hypothesis cannot be experimentally proven
Written by Grigory Rechistov in Uncategorized on 07.04.2024. Tags: hypothesis, testing,
TL;DR: there is a difference between “is proven” and “could not be disproven”.
Let’s say you have a hypothesis A. How do you test it? By doing a set of experiments E₁, E₂, E₃, etc., observing their outcomes and comparing them against predictions made by A.
How do …
Outcomes of tests
Written by Grigory Rechistov in Uncategorized on 23.01.2024. Tags: testing, flakiness, timeouts,
Patterns of behavior around tests
There are patterns in why and how tests fail and pass, and what corrective actions should and should not be applied by developers in response to that. Here are some do’s and don’t s for different good and bad situations.
We have a …
Test that times out has a bug in the test, not in the payload
Written by Grigory Rechistov in Uncategorized on 15.12.2021. Tags: testing, clean code, simulation,
TL;DR: Do not clump timeouts and other test failures together. Timeouts signal a problem with test harness design, not just a bug in the system it controls.
The software simulation field is unique in the following way. The input “data” our simulator program processes is actually “code” of some …