Currently going through a major rework. The MCTS/ARGA approach isn’t going to make much progress as it is, and looking for another level of abstraction. A lot of the code appearing here will be throwaway code from this point on, and only here to have a record of my thinking. I have pretty much given up on 2024 competition.
Working towards Kaggle’s Abstraction and Reasoning Challenge - Arc Prize 2024.
This is a fork of the ARGA project here github and paper.
Benchmarking on the 160 ARGA subset of ARC public training set (400).
| date | passed | ~last_run | ~arga_orig | ~gpar | ~icecuber | ~ale_2nd |
|---|---|---|---|---|---|---|
| 22 Aug | 22 | 3/35 | 6/63 | 10/59 | 9/60 | |
| 23 Aug | 23 | 2/1 | 4/35 | 7/63 | 9/57 | 9/59 |
| 24 Aug | 23 | 2/2 | 2/33 | 5/61 | 10/58 | 9/59 |
| 25 Aug | 25 | 3/1 | 4/33 | 7/61 | 12/58 | 9/57 |
| 28 Aug | 35 | 11/1 | 3/22 | 6/50 | 13/49 | 9/47 |
| 29 Aug | 40 | 5/0 | 5/19 | 7/46 | 16/47 | 10/43 |
| 30 Aug | 46 | 7/1 | 5/13 | 8/41 | 18/43 | 12/39 |
| 3 Sept | 44 | 1/3 | 3/13 | 5/40 | 15/42 | 10/39 |
| 4 Sept | 45 | 1/0 | 3/12 | 5/39 | 16/42 | 10/38 |
| 6 Sept | 47 | 3/1 | 5/12 | 7/39 | 18/42 | 12/38 |
| 7 Sept | 51 | 5/1 | 8/11 | 8/36 | 19/39 | 12/34 |
| 9 Sept | 55 | 7/3 | 7/6 | 7/31 | 22/38 | 15/33 |
| 1 Oct | 58 | 4/1 | 9/5 | 9/30 | 25/38 | 16/31 |
~arga_orig : is the arga code from github repo ran for 30 minutes per task github
~gpar : taken from 81 solved tasks as per github, of which 79 was in original ARGA subset.
~icecuber : the Icecuber that won 2020 competition github
~ale_2nd : part of 2nd place in 2020 competition github
- The total passes for ~arga_orig = 54
- The total passes for ~gpar = 79
- The total passes for ~icecuber = 71
- The total passes for ~ale_2nd = 73
The ~last_run is the previous time McARGA was run, ie the previous row
- McARGA is run for 5 minutes per task.
- Each ~solver column has the format:
#unique passes for MCARGA / #unique passes for ~solver
The GPAR paper includes valuable supplementary information in the Appendix, detailing their transformations and abstractions. Their use of multiple objects in the selection/transformation process differs from the original ARGA approach, which replaces the concept of dynamic parameter binding. While I have developed a similar idea, and am currently working on its implementation, the insights from the GPAR paper are extremely helpful. And furthermore hopefully McARGA can achieve a comparable result.