Trying to assess the logical reasoning capability of AI systems is difficult. Exciting progress has occurred recently in the domain of mathematical reasoning.

Researchers at New York University and Google published an article in Nature on January 17, 2024 describing a AI system called AlphaGeometry that "solves complex geometry problems at a level approaching a human Olympiad gold-medalist" according to the Google DeepMind website. The system solved 25 of 30 problems while the average human gold medalist solved 25.9. These problems were designed to challenge top highschoolers and not professional mathematicians.

The innovative AI system uses a neurosymbolic approach which includes a large language model (using a digital neural network) and a rule-bound deduction engine. The researchers created synthetic training data for the language model.

Geometry questions are just part of the International Mathematical Olympiad. A 10 million dollar challenge fund has been established for the Artificial Intelligence-Mathematical Olympiad (AIMO Prize). "The fund intends to spur the open development of AI models that can reason mathematically, leading to the creation of a publicly-shared AI model capable of winning a gold medal in the International Mathematical Olympiad (IMO)."