// TOPIC STREAM

2 posts tagged with "Language Modeling"

Language modeling methods and evaluation.

posts: 2

Conceptual diagram representing token memory and comparison in an H3 state space layer

Research::Aug 19, 20267 min

H3: The Memory and Comparison Capabilities SSMs Missed in Language Modeling

Hungry Hungry Hippos decomposes the linguistic weaknesses of state space models into token recall and comparison, then explains how H3, FlashConv, and hybrid attention combinations address them.

AI Research Language Modeling SSM2 more topics

referenced

Mamba state-space flow selectively passing and forgetting information according to input tokens

Research::Aug 19, 20267 min

Mamba: Approaching Content-Based Reasoning with Selective State Spaces

This article explains why Mamba targets both long-sequence efficiency and language-modeling performance through selective SSMs that change state-space parameters based on the input and hardware-aware parallel algorithms.

AI Research Mamba SSM2 more topics

referenced