-
Physics of Language Models: Part 1, Context-Free Grammar
Paper • 2305.13673 • Published • 8 -
Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process
Paper • 2407.20311 • Published • 5 -
Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems
Paper • 2408.16293 • Published • 27 -
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Paper • 2309.14316 • Published • 10
Collections
Discover the best community collections!
Collections including paper arxiv:2404.05405
-
SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding
Paper • 2408.15545 • Published • 38 -
Controllable Text Generation for Large Language Models: A Survey
Paper • 2408.12599 • Published • 65 -
To Code, or Not To Code? Exploring Impact of Code in Pre-training
Paper • 2408.10914 • Published • 45 -
Automated Design of Agentic Systems
Paper • 2408.08435 • Published • 40
-
Physics of Language Models: Part 1, Context-Free Grammar
Paper • 2305.13673 • Published • 8 -
Physics of Language Models: Part 3.2, Knowledge Manipulation
Paper • 2309.14402 • Published • 8 -
Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
Paper • 2404.05405 • Published • 11 -
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Paper • 2309.14316 • Published • 10
-
SuperCorrect: Supervising and Correcting Language Models with Error-Driven Insights
Paper • 2410.09008 • Published • 17 -
answerdotai/ModernBERT-base
Fill-Mask • 0.1B • Updated • 4.06M • 1.12k -
answerdotai/ModernBERT-large
Fill-Mask • 0.4B • Updated • 742k • 498 -
microsoft/phi-4
Text Generation • 15B • Updated • 380k • • 2.32k
-
Compression Represents Intelligence Linearly
Paper • 2404.09937 • Published • 28 -
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
Paper • 2404.06395 • Published • 24 -
Long-context LLMs Struggle with Long In-context Learning
Paper • 2404.02060 • Published • 36 -
Are large language models superhuman chemists?
Paper • 2404.01475 • Published • 17
-
SELF: Language-Driven Self-Evolution for Large Language Model
Paper • 2310.00533 • Published • 2 -
GrowLength: Accelerating LLMs Pretraining by Progressively Growing Training Length
Paper • 2310.00576 • Published • 2 -
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity
Paper • 2305.13169 • Published • 4 -
Transformers Can Achieve Length Generalization But Not Robustly
Paper • 2402.09371 • Published • 14
-
Physics of Language Models: Part 1, Context-Free Grammar
Paper • 2305.13673 • Published • 8 -
Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process
Paper • 2407.20311 • Published • 5 -
Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems
Paper • 2408.16293 • Published • 27 -
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Paper • 2309.14316 • Published • 10
-
SuperCorrect: Supervising and Correcting Language Models with Error-Driven Insights
Paper • 2410.09008 • Published • 17 -
answerdotai/ModernBERT-base
Fill-Mask • 0.1B • Updated • 4.06M • 1.12k -
answerdotai/ModernBERT-large
Fill-Mask • 0.4B • Updated • 742k • 498 -
microsoft/phi-4
Text Generation • 15B • Updated • 380k • • 2.32k
-
SciLitLLM: How to Adapt LLMs for Scientific Literature Understanding
Paper • 2408.15545 • Published • 38 -
Controllable Text Generation for Large Language Models: A Survey
Paper • 2408.12599 • Published • 65 -
To Code, or Not To Code? Exploring Impact of Code in Pre-training
Paper • 2408.10914 • Published • 45 -
Automated Design of Agentic Systems
Paper • 2408.08435 • Published • 40
-
Compression Represents Intelligence Linearly
Paper • 2404.09937 • Published • 28 -
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
Paper • 2404.06395 • Published • 24 -
Long-context LLMs Struggle with Long In-context Learning
Paper • 2404.02060 • Published • 36 -
Are large language models superhuman chemists?
Paper • 2404.01475 • Published • 17
-
Physics of Language Models: Part 1, Context-Free Grammar
Paper • 2305.13673 • Published • 8 -
Physics of Language Models: Part 3.2, Knowledge Manipulation
Paper • 2309.14402 • Published • 8 -
Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
Paper • 2404.05405 • Published • 11 -
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Paper • 2309.14316 • Published • 10
-
SELF: Language-Driven Self-Evolution for Large Language Model
Paper • 2310.00533 • Published • 2 -
GrowLength: Accelerating LLMs Pretraining by Progressively Growing Training Length
Paper • 2310.00576 • Published • 2 -
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity
Paper • 2305.13169 • Published • 4 -
Transformers Can Achieve Length Generalization But Not Robustly
Paper • 2402.09371 • Published • 14