---
title: "Go's UTF-8 Identifier Limitation"
date: 2024-11-12
tags: ["go"]
description: "Where Go's UTF-8 support stops: what works in identifiers, and what non-Latin scripts still can't do."
---

I've been exploring Go's UTF-8 support and was curious about how well it handles non-Latin scripts in code.

Go source files are UTF-8 encoded by default. You can use Unicode characters in variable names and function names. Chinese characters work fine:

```go
package main

import "fmt"

func main() {
    消息 := "Hello, World!"
    fmt.Println(消息)
}
```

## Attempting to Use Tamil Identifiers

I tried Tamil, my mother tongue:

```go
package main

import "fmt"

func main() {
    எண்ணிக்கை := 42
    fmt.Println("Value:", எண்ணிக்கை)
}
```

But the compiler throws errors:

```
./prog.go:6:11: invalid character U+0BCD '்' in identifier
```

### Why Combining Marks Fail

Tamil is an abugida. Consonant-vowel sequences are written as units with combining marks. Go's spec allows Unicode letters in identifiers but excludes combining marks (categories `Mn`, `Mc`, `Me`).

```mermaid
graph LR
    A["Tamil syllable"] --> B["Base consonant\n(Lo: valid)"]
    A --> C["Vowel sign\n(Mc: continuation only)"]
    B --> D["Visually incomplete\nalone"]
    C --> D
```

### Chinese vs Tamil

Chinese characters are classified under "Letter, Other" (`Lo`) -- standalone symbols. Tamil requires combining marks, which Go's spec explicitly excludes.

> Go's creators primarily aimed for consistent string handling through UTF-8 support. They didn't necessarily intend for native-language coding in identifiers requiring combining marks.