|
Dissociating formal and functional linguistic competence in large language models| title | Dissociating formal and functional linguistic competence in large language models |
|---|
| start_date | 2023/09/13 |
|---|
| schedule | 17h-18h |
|---|
| online | no |
|---|
| location_info | via Zoom |
|---|
| summary | Today’s large language models (LLMs) routinely generate coherent, grammatical and seemingly meaningful paragraphs of text. This achievement has led to speculation that LLMs have become “thinking machines”, capable of performing tasks that require reasoning and/or world knowledge. In this talk, I will introduce a distinction between formal competence—knowledge of linguistic rules and patterns—and functional competence—understanding and using language in the world. This distinction is grounded in human neuroscience, which shows that formal and functional competence recruit different cognitive mechanisms. I will show that the word-in-context prediction objective has allowed LLMs to essentially master formal linguistic competence; however, LLMs still lag behind at many aspects of functional linguistic competence, and improvements in this domain often depend on specialized fine-tuning or coupling with an external module. In the last part of the talk, I will present a case study highlighting the difficulties of disentangling formal and functional competence when it comes to evaluating world knowledge, and show that similar difficulties are present in neuroscience research. I will conclude by discussing the value of the formal/functional competence framework for evaluating and building flexible, humanlike models of language use. |
|---|
| responsibles | Bernard |
|---|
Workflow history| from state (1) | to state | comment | date |
|---|
| submitted | published | | 2023/10/11 15:03 UTC |
| |
|