Semantic Phase Transitions in Transformer Observation Geometries

Abstract

Each transformer layer is treated as an observation geometry obtained from hidden representations under a natural Euclidean metric and an empirical measure induced by a prompt distribution.

Keywords

• large language models
• transformer
• semantic phase transitions
• observation geometry
• random geometric graphs
• attention-based random connection model
• representation geometry
• in-context learning
• percolation
• scaling-laws
• emergent behaviour

Identifiers and source records

• DOI
• Works entry
• ORCID