1.8 KiB
— title: Marcus (2020): The Case Against Pure Deep Learning type: reference tags: :passepartout:architecture: —
Marcus (2020): The Case Against Pure Deep Learning
Gary Marcus's "The Next Decade in AI" argues that deep learning alone is "data hungry, shallow, brittle, and limited in its ability to generalize." The paper demonstrates GPT-2 failing at basic commonsense reasoning:
- "Yesterday I dropped my clothes off at the dry cleaners and have yet to pick them up. Where are my clothes?" → GPT-2: "at my mom's house."
- "There are six frogs on a log. Two leave, but three join. The number of frogs on the log is now" → GPT-2: "seventeen."
Marcus proposes four steps toward robust AI: hybrid architecture (combining neural and symbolic), large-scale knowledge (abstract and causal, not just statistical), reasoning (formal inference over structured representations), and cognitive models (frameworks for how entities relate). Passepartout implements all four: the perceive-reason-act pipeline is hybrid, the symbolic index is causal knowledge, Screamer + ACL2 provide reasoning, and the gate-bootstrapped ontology plus MOMo modules provide cognitive models.
Marcus's core claim — "we have no hope of achieving robust intelligence without first developing systems with deep understanding" — is the justification for Passepartout's entire neurosymbolic investment. The alternative is a system that works "on a good day" and fails unpredictably. The deterministic gate stack and Screamer admission gate are the engineering realization of Marcus's call for robustness.
Reference: Marcus, G. (2020). The Next Decade in AI: Four Steps Towards Robust Artificial Intelligence. arXiv:2002.06177.