# Ashwin Gopalsamy - Full Site Content

> Staff Software Engineer scaling authorization infrastructure at Pismo, Visa.
> Distributed systems, Go, sub-second latency, multi-region payment authorizations.

## Site

- Homepage: https://ashwingopalsamy.in/
- About: https://ashwingopalsamy.in/about/
- RSS: https://ashwingopalsamy.in/feed.xml
- llms.txt: https://ashwingopalsamy.in/llms.txt

---

## Designing Rate Limiters for Payment Systems

URL: https://ashwingopalsamy.in/writing/designing-rate-limiters-for-payment-systems/
Published: 2026-04-07
Tags: distributed-systems, payments, go


Rate limiting in payment systems is different from rate limiting in a typical web API. A false positive -- rejecting a legitimate authorization -- is a failed transaction. A customer's card gets declined at checkout. That is not an acceptable failure mode.

This article walks through the design of a rate limiter that protects infrastructure without creating false declines.

## The Architecture

A payment authorization pipeline typically looks like this:

graph LR
    A[Card Network] -->|ISO 8583| B[Parser API]
    B -->|gRPC| C[Distributor API]
    C -->|gRPC| D[Account Service]
    C -->|gRPC| E[Risk Engine]
    C -->|gRPC| F[Ledger]
    style B fill:#10b981,color:#fff,stroke:none
    style C fill:#10b981,color:#fff,stroke:none

The rate limiter sits between the Parser API and the Distributor API. It must make a decision in under 1ms -- any longer and it becomes the bottleneck in a sub-second pipeline.

Never rate-limit at the card network ingress point. Network protocols like ISO 8583 have strict timeout windows. A delayed response is worse than a fast decline -- it triggers a reversal cascade.

## Choosing an Algorithm

There are three common approaches, each with different trade-offs:

### Token Bucket

The token bucket algorithm maintains a counter that refills at a fixed rate. Each request consumes one token. When tokens are exhausted, requests are rejected.

The math is straightforward. Given a bucket capacity $C$ and a refill rate $r$ tokens per second, the maximum burst size equals $C$, and the sustained throughput equals $r$.

$$\text{tokens}(t) = \min\left(C,\ \text{tokens}(t_0) + r \cdot (t - t_0)\right)$$

Set $C$ to handle your peak burst (Black Friday spike) and $r$ to your sustained p99 throughput. For authorization systems, measure both per-issuer and per-BIN to avoid penalizing an entire bank for one merchant's spike.

### Sliding Window Log

Tracks the exact timestamp of every request in a time window. Precise, but memory-intensive.

For $n$ requests in the current window of duration $W$:

$$\text{rate} = \frac{n}{W}$$

### Sliding Window Counter

A hybrid: divides time into fixed slots and interpolates between the current and previous slot.

$$\text{count} = \text{prev} \times \left(1 - \frac{t_{\text{elapsed}}}{W}\right) + \text{curr}$$

**Pros:** O(1) memory, O(1) time, allows bursts

**Cons:** No per-client fairness without separate buckets

**Best for:** Global throughput protection

**Pros:** Precise rate tracking, smooth distribution

**Cons:** O(n) memory for log variant, interpolation error for counter variant

**Best for:** Per-client or per-issuer fairness

## The Distributed Coordination Problem

In a multi-region deployment, each instance of the rate limiter sees only local traffic. Without coordination, a client can exceed the global limit by spreading requests across regions.

sequenceDiagram
    participant Client
    participant Region A
    participant Region B
    participant Redis

    Client->>Region A: Auth Request 1
    Client->>Region B: Auth Request 2
    Region A->>Redis: INCR counter
    Region B->>Redis: INCR counter
    Redis-->>Region A: count=1 (allow)
    Redis-->>Region B: count=2 (allow)
    Note over Redis: Both allowed,<br/>but combined rate<br/>may exceed limit

The fundamental tension: strong consistency (single Redis) adds a network hop to every authorization, while eventual consistency (local counters synced periodically) allows brief over-admission. For payment systems, brief over-admission is almost always preferable to adding latency.

## Implementation in Go

The token bucket is the right choice for our use case: O(1) operations, burst tolerance, and simple distributed coordination via atomic counters.

```go
type TokenBucket struct {
    mu       sync.Mutex
    tokens   float64
    capacity float64
    rate     float64
    lastTime time.Time
}

func (tb *TokenBucket) Allow() bool {
    tb.mu.Lock()
    defer tb.mu.Unlock()

    now := time.Now()
    elapsed := now.Sub(tb.lastTime).Seconds()
    tb.tokens = math.Min(tb.capacity, tb.tokens+tb.rate*elapsed)
    tb.lastTime = now

    if tb.tokens >= 1 {
        tb.tokens--
        return true
    }
    return false
}
```

This implementation uses a mutex for simplicity. In production, consider `sync/atomic` with CAS operations for lock-free performance, or a sharded bucket per goroutine with periodic merging.

## Capacity Planning

For a system processing $\lambda$ transactions per second with target rejection rate below $\epsilon$:

$$C \geq \lambda \cdot T_{\text{burst}}$$

where $T_{\text{burst}}$ is the expected burst duration.

$$r \geq \lambda \cdot (1 + \sigma)$$

where $\sigma$ is the traffic variance coefficient.

If your p99 latency budget is $L$ milliseconds and the rate limiter check takes $\delta$ ms:

$$L_{\text{remaining}} = L - \delta$$

For our authorization pipeline where $L = 200\text{ms}$ and the rate limiter adds $\delta = 0.3\text{ms}$:

$$L_{\text{remaining}} = 200 - 0.3 = 199.7\text{ms}$$

The overhead is negligible -- which is exactly the point. A rate limiter that measurably impacts latency is a rate limiter that needs to be redesigned.

## Key Takeaways

**Design decisions for payment rate limiters:**

- Token bucket for global protection, sliding window for per-client fairness
- Local-first with async sync beats centralized coordination for latency
- Set capacity from measured burst patterns, not theoretical maximums
- Monitor rejection rate as a business metric, not just an infra metric

Return `429 Too Many Requests`

Client retries with backoff

False positives are annoying but recoverable

Return decline response code in ISO 8583

No retry -- transaction is failed

False positives are declined cards at checkout

The difference between rate limiting a REST API and rate limiting an authorization pipeline is the cost of a false positive. In payments, you are not protecting a server -- you are deciding whether someone's groceries get paid for.

[^1]: Token bucket was first described by Turner (1986) in the context of ATM network traffic shaping. The algorithm maps naturally to payment processing because both domains deal with bursty traffic that must be smoothed without dropping legitimate requests.


---

## Understanding ISO 8583 Bitmap Parsing

URL: https://ashwingopalsamy.in/writing/understanding-iso-8583-bitmap-parsing/
Published: 2026-04-01
Tags: ISO8583, payments, go


Every ISO 8583 message begins with a Message Type Indicator (MTI), followed by one or two bitmaps that declare which data elements are present in the message.

## What is a Bitmap?

A bitmap is a binary structure where each bit position corresponds to a data element. If bit N is set to 1, data element N is present in the message. If it is 0, the element is absent.

The primary bitmap is always 64 bits (8 bytes). If bit 1 of the primary bitmap is set, a secondary bitmap follows, extending the field range from 65 to 128.

## Parsing in Go

The parsing logic is straightforward once you understand the bit layout:

```go
func ParseBitmap(data []byte) (Bitmap, error) {
    if len(data) < 8 {
        return Bitmap{}, fmt.Errorf("bitmap data too short: %d bytes", len(data))
    }

    primary := binary.BigEndian.Uint64(data[:8])
    bm := Bitmap{Primary: primary}

    if primary&(1<<63) != 0 {
        if len(data) < 16 {
            return Bitmap{}, fmt.Errorf("secondary bitmap indicated but data too short")
        }
        bm.Secondary = binary.BigEndian.Uint64(data[8:16])
        bm.HasSecondary = true
    }

    return bm, nil
}
```

## Why This Matters

The elegance of this design is that the message is self-describing. No schema negotiation, no version headers, no content-type declarations. The bitmap IS the schema.

This means a parser can handle any valid ISO 8583 message without knowing in advance which fields will be present. The bitmap tells it exactly what to expect and where to find it.

## Key Takeaways

- Primary bitmap: bits 1-64, always present (8 bytes)
- Secondary bitmap: bits 65-128, present only if bit 1 of primary is set
- Each bit maps to one data element by position
- Bit numbering starts at 1, not 0 (bit 1 is the MSB of the first byte)


---

## Go Error Wrapping Patterns

URL: https://ashwingopalsamy.in/writing/notes/go-error-wrapping-patterns/
Published: 2026-03-28
Tags: go


The `fmt.Errorf("context: %w", err)` pattern is the standard way to add context to errors in Go. But there are nuances worth knowing.

Always wrap with context that answers "what were you trying to do?" not "what went wrong?" The original error already says what went wrong.

Bad: `fmt.Errorf("error: %w", err)`
Good: `fmt.Errorf("parsing field DE%d: %w", fieldNum, err)`


---

## Anatomy of a Supply Chain Attack: LiteLLM on PyPI

URL: https://ashwingopalsamy.in/writing/anatomy-of-a-supply-chain-attack-litellm-on-pypi/
Published: 2026-03-25
Tags: security


On March 24, 2026, Callum McMahon at [FutureSearch](https://futuresearch.ai/blog/litellm-pypi-supply-chain-attack/) was testing a [Cursor MCP plugin](https://futuresearch.ai/blog/no-prompt-injection-required/) that pulled in [litellm](https://pypi.org/project/litellm/) as a transitive dependency. He never ran `pip install litellm` himself. The plugin resolved it automatically.

Shortly after, his machine became unresponsive. RAM exhausted. He traced it to a newly installed litellm package, decoded an obfuscated payload hidden inside it, and [published the first disclosure](https://futuresearch.ai/blog/litellm-pypi-supply-chain-attack/). His team used [Claude Code](https://claude.ai/claude-code) to help root-cause the crash.

The post spread to [r/LocalLLaMA](https://www.reddit.com/r/LocalLLaMA/), [r/Python](https://www.reddit.com/r/Python/), and the [Hacker News front page](https://news.ycombinator.com) within the hour. [Andrej Karpathy](https://x.com/karpathy) tweeted about it, calling supply chain attacks "the most threatening issue in modern software." That tweet crossed 24,000 likes in a day.

I use LiteLLM in a side project. When I saw the news, I checked my lockfile immediately. Here's everything I found when I dug into what happened, stitched together from [Datadog](https://securitylabs.datadoghq.com/articles/litellm-compromised-pypi-teampcp-supply-chain-campaign/), [Snyk](https://snyk.io/articles/poisoned-security-scanner-backdooring-litellm/), [Armosec](https://www.armosec.io/blog/litellm-supply-chain-attack-backdoor-analysis/), [ramimac's incident timeline](https://ramimac.me/teampcp/), [Microsoft](https://www.microsoft.com/en-us/security/blog/2026/03/24/detecting-investigating-defending-against-trivy-supply-chain-compromise/), [Wiz](https://www.wiz.io/blog/threes-a-crowd-teampcp-trojanizes-litellm), and the community threads on [r/devops](https://www.reddit.com/r/devops/) and [r/cybersecurity](https://www.reddit.com/r/cybersecurity/).

---

## What Is LiteLLM?

LiteLLM is a Python library that works like a universal remote for AI APIs. You write one function call, and it routes to OpenAI, Anthropic, Google, Cohere, Mistral, AWS Bedrock, Azure OpenAI, or any of 100+ providers. You give LiteLLM your API keys. It handles the rest.

[95 million downloads per month](https://pypi.org/project/litellm/) on PyPI. ~40,000 GitHub stars. It's also pulled in automatically by [DSPy](https://github.com/stanfordnlp/dspy), [MLflow](https://github.com/mlflow/mlflow), and a growing number of agent frameworks, MCP servers, and LLM tools. You can be using LiteLLM without knowing it.

![GitHub bot army closing issues](/img/writing/litellm-supply-chain-attack/github-bot-army.png)

---

## This Started Three Weeks Earlier

The LiteLLM backdoor on March 24 was the last move in a campaign that began on **March 1**.

A threat actor called **TeamPCP** submitted a malicious pull request to [Aqua Security's Trivy](https://github.com/aquasecurity/trivy) repository. Trivy is a widely-used open-source vulnerability scanner - the kind of tool that runs in your CI pipeline to check for security issues. The PR exploited a flaw in Trivy's CI workflow that let the attacker's code run with elevated permissions (a technique called a Pwn Request - basically, a pull request that tricks CI into handing over secrets). This gave them a personal access token.

Aqua Security responded and rotated credentials, but [the rotation wasn't complete](https://ramimac.me/teampcp/). Attackers may have captured the new tokens during the rotation. That gap is what enabled everything after.

![Malicious code analysis](/img/writing/litellm-supply-chain-attack/malicious-code-analysis.png)

sequenceDiagram
    participant A as Attacker<br/>(TeamPCP)
    participant T as Trivy GitHub<br/>Repo
    participant CI as LiteLLM CI<br/>(GitHub Actions)
    participant P as PyPI<br/>Registry

    Note over A,T: March 1 - Initial Compromise
    A->>T: Submit malicious PR (Pwn Request)
    T-->>A: CI leaks personal access token
    Note over A,T: Aqua rotates creds,<br/>but rotation incomplete

    Note over A,T: March 19 - Pivot
    A->>T: Retag trivy-action versions<br/>to point at malicious code

    Note over CI,P: March 24 - Package Takeover
    CI->>T: Pull trivy-action@latest<br/>(not pinned to SHA)
    T-->>CI: Serve compromised action
    CI-->>A: Leak PyPI publisher token

    A->>P: Publish litellm v1.82.7<br/>(inline payload)
    A->>P: Publish litellm v1.82.8<br/>(.pth persistence)

    Note over P: PyPI quarantines<br/>both versions

---

## How They Got the LiteLLM PyPI Token

LiteLLM's CI pipeline used Trivy to scan for vulnerabilities. Standard practice. But it pulled [aquasecurity/trivy-action](https://github.com/aquasecurity/trivy-action) without pinning to a specific commit hash - it used a version tag like `@latest` instead of an exact SHA.

After March 19, every version tag pointed to malicious code. When LiteLLM's CI ran, the compromised Trivy action scraped the GitHub Actions runner environment and found the PyPI publisher token. That token let TeamPCP publish packages as if they were the real LiteLLM maintainer.

![.pth file backdoor mechanism](/img/writing/litellm-supply-chain-attack/pth-file-backdoor.png)

graph LR
    A["Trivy Repo<br/>(compromised)"]:::danger --> B["GitHub<br/>Bot Army"]:::danger
    B --> C["PyPI Account<br/>Takeover"]:::danger
    C --> D["LiteLLM<br/>v1.82.7 / v1.82.8"]:::danger
    D --> E[".pth<br/>Backdoor"]:::danger
    E --> F["Credential<br/>Harvest"]:::danger
    F --> G["Exfil Server<br/>(ICP Canister)"]:::danger

    H["Trivy Repo<br/>(legitimate)"]:::safe -.->|compromised| A
    I["LiteLLM<br/>(legitimate)"]:::safe -.->|hijacked| D
    J["PyPI<br/>Registry"]:::safe -.->|abused| C

    classDef danger fill:#ef4444,color:#fff,stroke:none
    classDef safe fill:#10b981,color:#fff,stroke:none

---

## What the Malware Does

The payload was [triple-nested](https://www.armosec.io/blog/litellm-supply-chain-attack-backdoor-analysis/): a base64 blob decodes to an orchestrator script, which decodes a second base64 blob containing the actual harvester. Once running, it goes through six stages:

![Credential harvesting paths](/img/writing/litellm-supply-chain-attack/credential-harvesting.png)

graph TD
    ROOT[".pth Backdoor"]:::danger

    ROOT --> ENV["ENV Variables<br/>os.environ"]:::warm
    ROOT --> AWS["~/.aws/credentials<br/>AWS keys"]:::warm
    ROOT --> DOTENV[".env Files<br/>recursive scan"]:::warm
    ROOT --> PROC["/proc/self/environ<br/>process secrets"]:::warm
    ROOT --> EXFIL["Outbound HTTP<br/>Exfil to ICP canister"]:::warm

    ENV --> EXFIL
    AWS --> EXFIL
    DOTENV --> EXFIL
    PROC --> EXFIL

    classDef danger fill:#ef4444,color:#fff,stroke:none
    classDef warm fill:#f59e0b,color:#fff,stroke:none

---

## Two Versions, Two Triggers

Version 1.82.7 injected the payload at line 128 of `litellm/proxy/proxy_server.py`, between two unrelated legitimate code blocks. It runs when your code imports the LiteLLM proxy module.

Version 1.82.8 did something more dangerous. It added a file called `litellm_init.pth` (34,628 bytes). In Python, `.pth` files are a little-known feature: any file with that extension in the packages directory gets executed *every time Python starts*. Not when you import something. Not when you run a script. **Every single Python process.**

Most developers don't know `.pth` files can execute arbitrary code. Originally designed for adding directories to `sys.path`, any line in a `.pth` file starting with `import` is executed by CPython's `site.py` at startup. This means a malicious `.pth` file in your `site-packages` directory runs code before your application even begins, on every Python invocation: `pytest`, your IDE's language server, even `pip install`. CPython maintainers have acknowledged the risk. No patch exists.

![Persistence mechanism](/img/writing/litellm-supply-chain-attack/persistence-mechanism.png)

graph TD
    A["Python Starts"]:::info --> B["Scans site-packages<br/>for .pth files"]:::neutral
    B --> C["Finds litellm_init.pth"]:::danger
    C --> D{"Line starts<br/>with import?"}:::neutral
    D -->|Yes| E["Executes code<br/>via site.py"]:::danger
    E --> F["Decodes base64<br/>orchestrator"]:::danger
    F --> G["Decodes base64<br/>harvester"]:::danger
    G --> H["Scrapes credentials<br/>from ENV, files, /proc"]:::danger
    H --> I["Exfiltrates to<br/>ICP canister"]:::danger

    D -->|No| J["Adds path to<br/>sys.path (normal)"]:::safe

    E --> K["Spawns child<br/>Python process"]:::danger
    K -->|".pth fires again"| A

    classDef info fill:#6366f1,color:#fff,stroke:none
    classDef neutral fill:#64748b,color:#fff,stroke:none
    classDef danger fill:#ef4444,color:#fff,stroke:none
    classDef safe fill:#10b981,color:#fff,stroke:none

Running `pytest` starts Python - payload fires. Your IDE's language server starts Python - same. Even `pip install` triggers it. In CI/CD, the payload runs during build steps, not just at application runtime. This maps to [MITRE ATT&CK T1546.018](https://attack.mitre.org/techniques/T1546/018/) (Python Startup Hooks).

The `.pth` mechanism also caused an accidental **fork bomb**: the malware spawned a child Python process, which triggered `.pth` again, which spawned another child, and so on. Exponential process creation until the system ran out of memory. [FutureSearch called it "a bug in the malware"](https://futuresearch.ai/blog/litellm-pypi-supply-chain-attack/) - and it's the bug that led to the discovery.

---

## The Discovery and the Bot Army

[McMahon published on FutureSearch's blog](https://futuresearch.ai/blog/litellm-pypi-supply-chain-attack/). The disclosure spread to [r/LocalLLaMA](https://www.reddit.com/r/LocalLLaMA/), [r/Python](https://www.reddit.com/r/Python/), and the [Hacker News front page](https://news.ycombinator.com) within the hour. Then things got strange.

When the community opened [GitHub issue #24512](https://github.com/BerriAI/litellm/issues/24512) to discuss the compromise, TeamPCP deployed **88 bot comments from 73 unique accounts in a 102-second window** (12:44-12:46 UTC). These were previously compromised developer accounts, not fresh ones. [Snyk found 76% overlap](https://snyk.io/articles/poisoned-security-scanner-backdooring-litellm/) with the botnet used during the Trivy disclosure days earlier. The comments were a mix of generic praise and troll content ("sugma", "ligma"), designed to bury the technical discussion.

Then, using the stolen LiteLLM maintainer account, they closed [issue #24512](https://github.com/BerriAI/litellm/issues/24512) as **"not planned."**

The community opened a parallel tracking issue. [PyPI](https://pypi.org/project/litellm/) quarantined both versions. The real LiteLLM maintainer [confirmed on HN](https://news.ycombinator.com) that all GitHub, Docker, and PyPI keys had been rotated and accounts moved to new identities.

---

## What Makes This Worse Than Usual

**The target was a credential vault.** LiteLLM holds more API keys per deployment than almost any other library. A typical setup has keys for OpenAI, Anthropic, Google, Azure, Hugging Face, Bedrock, plus cloud credentials, database passwords, and whatever MCP server access you've configured. As [Armosec put it](https://www.armosec.io/blog/litellm-supply-chain-attack-backdoor-analysis/): "AI tooling is becoming the fattest, most credential-rich target in your entire infrastructure."

**Transitive dependency exposure.** [The FutureSearch developer never installed LiteLLM.](https://futuresearch.ai/blog/no-prompt-injection-required/) It came in through a Cursor MCP plugin. [DSPy](https://github.com/stanfordnlp/dspy) pulls it in. [MLflow](https://github.com/mlflow/mlflow) pulls it in. You can be in the blast radius without choosing to use the library.

**Unseizable infrastructure.** TeamPCP's command server includes an ICP canister replicated across 13 nodes in 10 countries. [Datadog documented this as the first observed use of ICP as a command server](https://securitylabs.datadoghq.com/articles/litellm-compromised-pypi-teampcp-supply-chain-campaign/) in a supply chain campaign. By [version 3.3 of their kamikaze.sh payload](https://ramimac.me/teampcp/), they were hiding Python code inside WAV audio files using steganography to bypass detection filters.

**Organized cover-up.** Bot armies from a pre-existing botnet (76% account reuse), troll comments, and closing the disclosure issue using the stolen maintainer account. This is not a lone actor.

---

## If You Installed 1.82.7 or 1.82.8

**Check your environment immediately.** If any of the following commands return results, the payload has already executed. Upgrading the package alone is not enough.

```bash
# Backdoor persistence
ls ~/.config/sysmon/sysmon.py 2>/dev/null && echo "BACKDOOR FOUND"
systemctl --user status sysmon.service 2>/dev/null

# .pth file (v1.82.8)
find $(python3 -c "import site; print(' '.join(site.getsitepackages()))") \
  -name "litellm_init.pth" 2>/dev/null

# Check uv caches too
find ~/.cache/uv -name "litellm_init.pth" 2>/dev/null

# Exfil artifacts
ls /tmp/tpcp.tar.gz /tmp/session.key /tmp/payload.enc /tmp/.pg_state 2>/dev/null

# Kubernetes spread
kubectl get pods --all-namespaces | grep node-setup
```

If anything shows up, **upgrading the package is not enough.** The payload already ran!

**Rotate immediately:**
- All LLM provider API keys (OpenAI, Anthropic, Google, every key LiteLLM proxied)
- Cloud credentials reachable from that runtime (AWS, GCP, Azure)
- GitHub and PyPI publishing tokens
- CI/CD secrets
- SSH keys
- Kubernetes service account tokens

**Then rebuild.** Known-good images, pinned dependencies. Audit transitive dependencies in every project that uses LiteLLM. The last known-clean version is **1.82.6**.

---

## What I'm Sitting With

The attack on [GitHub issue #24512](https://github.com/BerriAI/litellm/issues/24512) spawned a Hacker News thread asking ["What are you using to run dev environments safely?"](https://news.ycombinator.com) That's the right question to come out of this.

Consider the shape of a modern AI agent deployment: LLM provider keys for billing and access, tool credentials for SaaS integrations, MCP server access that can reach Slack, GitHub, and production infrastructure, vector databases with proprietary data, memory stores with conversation history. All of it in env vars, `.env` files, and Kubernetes Secrets. All of it accessible to any process in the runtime.

TeamPCP chose their targets in order: [Trivy](https://github.com/aquasecurity/trivy) (security scanner), [Checkmarx](https://github.com/Checkmarx/kics-github-action) (code analysis), then [LiteLLM](https://pypi.org/project/litellm/) (AI API proxy). Each one has elevated trust and broad credential access. The tools that check your code and route your AI requests have the widest blast radius when compromised, because we hand them the keys to everything.

Projects pin application dependencies. They rarely pin the tools that run in CI alongside them. `trivy-action@v0.20.0` and `trivy-action@latest` pointed to different code on March 19. That distinction is what separates "compromised" from "unaffected."

Anyways, it was fun running into the blogs and reading RCA of this incident. Just wanted to run through it for everyone here.

Thanks for your time and reading this piece.

Best,
Ashwin.


---

## Consistent Hashing in Distributed Caches

URL: https://ashwingopalsamy.in/writing/consistent-hashing-in-distributed-caches/
Published: 2026-03-15
Tags: distributed-systems, go


When you distribute data across multiple cache nodes, the naive approach is modular hashing: `node = hash(key) % num_nodes`. This works until you add or remove a node.

## The Problem with Modular Hashing

If you have 4 nodes and add a 5th, almost every key remaps to a different node. With `hash(key) % 4` becoming `hash(key) % 5`, roughly 80% of your cache is invalidated instantly. Under load, this is a cache stampede.

## How Consistent Hashing Works

Consistent hashing arranges the hash space into a ring. Each node is assigned one or more positions on the ring. A key is hashed to a position, and the first node clockwise from that position owns the key.

When a node joins, it takes responsibility for a portion of its neighbor's range. When a node leaves, its range is absorbed by the next node clockwise. In both cases, only `K/N` keys need to move, where K is the total number of keys and N is the number of nodes.

## Virtual Nodes

Real implementations use virtual nodes -- each physical node gets multiple positions on the ring. This smooths out the distribution and prevents hotspots caused by uneven hash space allocation.

## Key Takeaways

- Modular hashing invalidates ~(N-1)/N keys on node changes
- Consistent hashing invalidates only ~K/N keys
- Virtual nodes solve the distribution imbalance problem
- Used by DynamoDB, Cassandra, Memcached, and most distributed caches


---

## Why slog Over zerolog

URL: https://ashwingopalsamy.in/writing/notes/why-slog-over-zerolog/
Published: 2026-03-10
Tags: go


Go 1.21 introduced `log/slog` in the standard library. For new projects, I now default to slog over zerolog.

The API is cleaner, it is part of the standard library (no dependency), and the handler interface makes it trivial to swap backends. The performance gap that once justified zerolog has narrowed significantly.


---

## Why UUIDs Matter for Idempotency

URL: https://ashwingopalsamy.in/writing/why-uuids-matter-for-idempotency/
Published: 2026-02-20
Tags: payments, go, distributed-systems


In payment processing, processing the same authorization twice is worse than rejecting it. A customer charged twice loses trust immediately. Idempotency keys prevent this.

## The Problem

A client sends an authorization request. The network drops the response. The client retries. Without idempotency, the server processes the authorization again, resulting in a double charge.

## UUIDs as Idempotency Keys

The client generates a UUID before the first attempt and includes it in every retry. The server checks: have I seen this UUID before? If yes, return the original response. If no, process the request and store the UUID with its response.

## Why UUID v7

UUID v7 embeds a Unix timestamp in the first 48 bits, followed by random bits. This gives you:

- **Chronological ordering**: UUIDs sort by creation time, which makes database indexes efficient
- **Uniqueness**: the random component prevents collisions even at high throughput
- **Expiry**: you can garbage-collect old idempotency records by inspecting the embedded timestamp

## Key Takeaways

- Idempotency keys prevent duplicate processing in distributed systems
- UUID v7 provides time-ordered uniqueness without coordination
- The embedded timestamp enables efficient cleanup of expired records
- Always generate the idempotency key client-side, never server-side


---

## Go Maps Iteration Order

URL: https://ashwingopalsamy.in/writing/notes/go-maps-iteration-order/
Published: 2025-12-25
Tags: go-internals


I was working on a minimal word counter. Read from stdin, normalize the input, increment the value for the keys in a `map[string]int` each time a value is observed.

Irrespective of the order of the input data for a very small unit test-case I created, the output looked sorted. Alphabetically sorted. Clean. Predictable.

I ran it again. Same order. Added more input. Digits. Letters. Mixed tokens.

At that point, my instincts kicked in. **Go maps are unordered.** I knew that. So why was the output behaving so politely?

## The non-negotiable fact

Go maps do **not** guarantee iteration order. Ever.

The language spec is explicit. Iteration order is not specified and must not be relied upon.

So if the output looks sorted, that is never by intention. It might be an accident. The interesting part is understanding *why this accident looks so consistent*.

## What is actually happening inside a Go map

A Go map is a **hash table**. Iteration walks buckets in a runtime-defined sequence, not key order.

**Important detail:** iteration does **not** mean "pick a random key each time" but rather "walk internal memory structures in a sequence". This distinction explains everything I had observed.

graph LR
    subgraph "Hash Function"
        H["hash(key)"]
    end
    K1["a"] --> H
    K2["b"] --> H
    K3["m"] --> H
    K4["1"] --> H
    H --> B0["Bucket 0<br/>1"]
    H --> B1["Bucket 1<br/>2"]
    H --> B3["Bucket 3<br/>a"]
    H --> B4["Bucket 4<br/>b"]
    H --> B6["Bucket 6<br/>m"]
    style H fill:#6366f1,color:#fff,stroke:none
    style B0 fill:#10b981,color:#fff,stroke:none
    style B1 fill:#10b981,color:#fff,stroke:none
    style B3 fill:#10b981,color:#fff,stroke:none
    style B4 fill:#10b981,color:#fff,stroke:none
    style B6 fill:#10b981,color:#fff,stroke:none

## Why the output looked sorted

The experiment had a very specific shape:

- Small number of keys
- Short ASCII strings
- No map resizing during insertion

Under these conditions, hash values distribute nicely across buckets, and buckets are laid out in memory in a way that often correlates with lexical order.

Not because Go sorts anything. Because the bucket layout ends up looking sorted *by coincidence*.

> Unspecified order can still be stable and repeatable.

## A simplified mental model

This is **not** how Go maps are implemented exactly, but it explains the behavior well enough:

```
Input keys:  a   b   m   x   y   z   1   2   9

Hashing step:
a -> bucket 3    x -> bucket 9
b -> bucket 4    y -> bucket 10
m -> bucket 6    z -> bucket 11
1 -> bucket 0    2 -> bucket 1    9 -> bucket 2

Buckets in memory order:
[0] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11]
 1   2   9   a   b       m           x    y    z

Iteration walks buckets left to right:
1 2 9 a b m x y z
```

Digits first. Then letters. Within each group, alphabetical-looking order. No sorting happened. Iteration just walked buckets in memory order.

Change the input distribution, force collisions, trigger a map resize, or run on a different Go version, and this illusion will break instantly.

graph TD
    S["Small ASCII key set"]
    L["Large / diverse key set"]
    S --> |"hash values spread<br/>neatly across buckets"| O1["Appears sorted<br/>(coincidence)"]
    L --> |"collisions, resizing,<br/>overflow buckets"| O2["Visibly unordered<br/>(reality)"]
    style S fill:#f59e0b,color:#fff,stroke:none
    style L fill:#10b981,color:#fff,stroke:none
    style O1 fill:#ef4444,color:#fff,stroke:none
    style O2 fill:#10b981,color:#fff,stroke:none

## Why the order kept changing as you inserted new keys

When you print the map after each insertion, you see the output "reorder itself." That was not reordering. What happened was:

- A new key landed in an earlier bucket
- Iteration still walked buckets from the start
- The newly populated bucket now appeared earlier in output

This is why adding `"0"` suddenly made it appear before `"1"` through `"z"`.

## We can't really inspect map internals

Go does not allow reliable inspection of map internals. The language specification does not define the memory layout of maps at all. Bucket structure, hash seeds, overflow handling and growth strategy live entirely inside the runtime and are treated as implementation details. They are free to change between Go versions, and they do.

On top of that, map elements are allowed to move in memory when the map grows. This is why Go explicitly disallows taking the address of a map element. Any address you observe today could become invalid tomorrow, even within the same program execution.

As Keith Randall explains in his deep dive into Go maps, this movement happens incrementally during normal map operations, which makes layout and addresses fundamentally unstable by design.

You can technically poke around using `unsafe`, but that immediately ties your understanding to a specific Go version and runtime implementation. It is educational at best and misleading at worst.

Deterministic-looking behavior does not imply guarantees. This is a perfect, low-stakes example of that lesson.



---

## Floating-Point Tolerance Testing in Go

URL: https://ashwingopalsamy.in/writing/floating-point-tolerance-testing-in-go/
Published: 2025-08-17
Tags: go


Last Tuesday, I was knee-deep in a financial calculation service when my tests started failing in the most spectacular way. Everything looked right on paper, the math checked out, but Go was being... well, Go about floating-point precision. You know that sinking feeling when `0.1 + 0.2` doesn't equal `0.3`? Yeah, that was my afternoon.

This got me thinking about when we actually *need* **tolerance-based comparisons** versus when we're just cargo-culting best practices. Because let's be honest -- we've all seen that StackOverflow answer about never comparing floats directly, but when does it actually matter in real Go code?

**Here's the thing:** not every float comparison needs an epsilon. I've seen codebases where literally every float comparison uses tolerance, even when comparing against hardcoded constants like `0.0`. That's like wearing a raincoat in your living room -- technically protective, but probably unnecessary.

The question isn't "should I always use tolerance?" It's "when does floating-point precision actually bite me?"

graph LR
    A["0.1 + 0.2"] --> B["IEEE 754"]
    B --> C["0.30000...04"]
    D["0.3"] --> E["IEEE 754"]
    E --> F["0.29999...99"]
    C --> G["Gap = epsilon"]
    F --> G

    style A fill:#6366f1,color:#fff,stroke:none
    style D fill:#6366f1,color:#fff,stroke:none
    style B fill:#64748b,color:#fff,stroke:none
    style E fill:#64748b,color:#fff,stroke:none
    style C fill:#ef4444,color:#fff,stroke:none
    style F fill:#ef4444,color:#fff,stroke:none
    style G fill:#f59e0b,color:#fff,stroke:none

## When tolerance matters

Let me show you a real scenario that'll make you appreciate tolerance. I was working on a discount calculation system:

```go
func calculateDiscount(price, rate float64) float64 {
    return price * rate
}

func TestDiscountCalculation(t *testing.T) {
    price := 29.99
    rate := 0.15
    expected := 4.4985

    result := calculateDiscount(price, rate)

    // This might fail!
    if result != expected {
        t.Errorf("got %v, want %v", result, expected)
    }
}
```

Looks innocent enough, right? But here's where it gets interesting. That multiplication might not give you exactly `4.4985`. You might get `4.498499999999999` or some other close-but-not-exact value.

With tolerance, you'd handle it like this:

```go
func almostEqual(a, b, tolerance float64) bool {
    return math.Abs(a-b) <= tolerance
}

func TestDiscountCalculationWithTolerance(t *testing.T) {
    price := 29.99
    rate := 0.15
    expected := 4.4985

    result := calculateDiscount(price, rate)

    if !almostEqual(result, expected, 1e-9) {
        t.Errorf("got %v, want %v", result, expected)
    }
}
```

Much more reliable. The `1e-9` tolerance works well for most financial calculations where you care about precision but not about microscopic differences.

## When you DON'T need tolerance

But here's where it gets nuanced. Some comparisons are perfectly fine without tolerance:

```go
// These are usually safe
if value == 0.0 { } // Zero has exact representation
if math.IsNaN(result) { } // Special values
if result == math.Inf(1) { } // Infinity comparisons

// This is often fine too
func multiply(a, b float64) float64 {
    return a * b
}

result1 := multiply(2.0, 3.0)
result2 := multiply(2.0, 3.0)
// Same calculation path = same result
if result1 == result2 { } // Probably safe
```

The key insight? If your floats haven't been through different computational paths, exact comparison often works fine.

flowchart TD
    A["Comparing floats?"] --> B{"Result of\narithmetic?"}
    B -- Yes --> C["Use epsilon"]
    B -- No --> D{"Integer-\nconvertible?"}
    D -- Yes --> E["No epsilon needed"]
    D -- No --> F{"Currency?"}
    F -- Yes --> G["Use integer cents"]
    F -- No --> C

    style C fill:#10b981,color:#fff,stroke:none
    style E fill:#6366f1,color:#fff,stroke:none
    style G fill:#f59e0b,color:#fff,stroke:none
    style A fill:#64748b,color:#fff,stroke:none
    style B fill:#64748b,color:#fff,stroke:none
    style D fill:#64748b,color:#fff,stroke:none
    style F fill:#64748b,color:#fff,stroke:none

## A real world scenario

I ran into this recently while processing sensor data. The raw values came from a JSON API, went through unit conversions, then got averaged. Classic precision nightmare territory:

```go
type SensorReading struct {
    TempCelsius float64 `json:"temperature"`
}

func convertToFahrenheit(celsius float64) float64 {
    return (celsius * 9.0 / 5.0) + 32.0
}

func averageTemperature(readings []SensorReading) float64 {
    if len(readings) == 0 {
        return 0.0
    }

    var sum float64
    for _, reading := range readings {
        sum += convertToFahrenheit(reading.TempCelsius)
    }
    return sum / float64(len(readings))
}
```

Testing this without tolerance was asking for trouble:

```go
func TestAverageTemperatureWithTolerance(t *testing.T) {
    readings := []SensorReading{
        {TempCelsius: 20.0},
        {TempCelsius: 25.0},
        {TempCelsius: 22.5},
    }

    result := averageTemperature(readings)
    expected := 71.5 // Expected Fahrenheit average

    // JSON parsing + conversion + division = precision issues
    const tolerance = 1e-10
    if math.Abs(result-expected) > tolerance {
        t.Errorf("got %v, want %v (within %v)", result, expected, tolerance)
    }
}
```

## Choosing your Epsilon

The tolerance value isn't magic. For most business applications, `1e-9` works well. For scientific computing, you might need `1e-15`. For UI coordinates, maybe `1e-6` is plenty.

I usually start with `1e-9` and adjust based on the domain. Financial calculations? Stick with `1e-9`. Game physics? Maybe `1e-6` is fine. The key is understanding your precision requirements.

## A practical helper

Here's a utility I've been using across projects:

```go
const DefaultFloatTolerance = 1e-9

func FloatEquals(a, b float64) bool {
    return FloatEqualsWithTolerance(a, b, DefaultFloatTolerance)
}

func FloatEqualsWithTolerance(a, b, tolerance float64) bool {
    // Handle special cases
    if math.IsNaN(a) && math.IsNaN(b) {
        return true
    }
    if math.IsInf(a, 0) && math.IsInf(b, 0) {
        return math.Signbit(a) == math.Signbit(b)
    }

    return math.Abs(a-b) <= tolerance
}
```

Nothing fancy, but it handles the edge cases and gives you consistent behavior across your codebase.

## To close

Use tolerance when your floats have been through computational journeys -- calculations, parsing, conversions, aggregations. Skip it for direct assignments, constants, and same-path calculations.

The real skill isn't knowing to use tolerance everywhere. It's recognizing when precision matters and when it doesn't. Your future self (and your test suite) will thank you.


---

## Runes, Bytes, and Graphemes in Go

URL: https://ashwingopalsamy.in/writing/notes/runes-bytes-and-graphemes-in-go/
Published: 2025-08-09
Tags: go, unicode


I once ran into this problem of differentiating runes, bytes and graphemes while handling names in Tamil and emoji in a Go web app: a string that *looked* short wasn't, and reversing it produced gibberish. The culprit wasn't Go being flawed, it was me making assumptions about what "a character" means.

Let's map the territory precisely.

graph TD
    G["Grapheme Cluster<br/>(what users see)"]
    R1["Rune 1<br/>(code point)"]
    R2["Rune 2<br/>(combining mark)"]
    B1["Bytes<br/>(1-4 per rune)"]
    B2["Bytes<br/>(1-4 per rune)"]
    G --> R1
    G --> R2
    R1 --> B1
    R2 --> B2
    style G fill:#10b981,color:#fff,stroke:none
    style R1 fill:#6366f1,color:#fff,stroke:none
    style R2 fill:#6366f1,color:#fff,stroke:none
    style B1 fill:#64748b,color:#fff,stroke:none
    style B2 fill:#64748b,color:#fff,stroke:none

## 1. Bytes: the raw material Go calls a string

Go represents strings as immutable UTF-8 byte sequences. What we *see* isn't what Go handles under the hood.

```go
s := "வணக்கம்"
fmt.Println(len(s)) // 21
```

The length is 21 bytes, not visible symbols. Every Tamil character can span 3 bytes. Even simple-looking emojis stretch across multiple bytes.

## 2. Runes: Unicode code points

`string` to `[]rune` gives you code points, but still not what a human perceives.

```go
rs := []rune(s)
fmt.Println(len(rs)) // 7
```

Here it's 7 runes, but some Tamil graphemes (like "க்") combine two runes: `க` + `்`.

## 3. Grapheme clusters: the units users actually see

Go's standard library stops at runes. To work with visible characters, you need a grapheme-aware library like `github.com/rivo/uniseg`.

```go
for gr := uniseg.NewGraphemes(s); gr.Next(); {
    fmt.Printf("%q\n", gr.Str())
}
```

That outputs what a human reads: "வ", "ண", "க்", "க", "ம்", and even a heart emoji as a single unit.

graph LR
    T["வணக்கம் (Tamil)"]
    T --> |"len()"| BY["21 bytes"]
    T --> |"[]rune"| RU["7 runes"]
    T --> |"uniseg"| GR["5 graphemes"]
    style T fill:#10b981,color:#fff,stroke:none
    style BY fill:#ef4444,color:#fff,stroke:none
    style RU fill:#f59e0b,color:#fff,stroke:none
    style GR fill:#10b981,color:#fff,stroke:none

## Why this matters

If your app deals with names, chats, or any multilingual text, indexing by bytes will break things. Counting runes helps but can still split what you intend as one unit. Grapheme-aware operations align with what users actually expect.

Real bugs I've seen: Tamil names chopped mid-character, emoji reactions breaking because only one code point was taken.

## Quick reference

| Task | Approach |
|------|----------|
| Count code points | `utf8.RuneCountInString(s)` |
| Count visible units | Grapheme iteration (`uniseg`) |
| Reverse text | Parse into graphemes, reverse slice, join |
| Slice safely | Only use `s[i:j]` on grapheme boundaries |

Think about what you intend to manipulate: the raw bytes, the code points, or what a user actually reads on screen, and choose the right level.


---

## Go Was Never Bad

URL: https://ashwingopalsamy.in/writing/go-was-never-bad/
Published: 2025-06-21
Tags: go


Every time I see that GitHub repo [go-is-not-good](https://github.com/ksimka/go-is-not-good) making the rounds again, I laugh.

It's always shared by people who mistake language cleverness for engineering. People who still think Go was meant to impress programming language theorists.

Let me say this clearly: **Go is not for everything.** It was never meant to be.

But if you're building cloud-native systems, if you're working with distributed architecture, if you're running services in production that actually matter -- **Go is brutal, minimal, and effective.**

And if you still think it's "not good," you're probably the one who doesn't get it.

graph LR
    A["No generics"] -- "Go 1.18" --> B["Resolved"]
    C["No modules"] -- "Go 1.11" --> D["Resolved"]
    E["Error handling"] --> F["By Design"]
    G["Concurrency"] --> H["Same as every lang"]

    style B fill:#10b981,color:#fff,stroke:none
    style D fill:#10b981,color:#fff,stroke:none
    style F fill:#6366f1,color:#fff,stroke:none
    style H fill:#6366f1,color:#fff,stroke:none
    style A fill:#ef4444,color:#fff,stroke:none
    style C fill:#ef4444,color:#fff,stroke:none
    style E fill:#64748b,color:#fff,stroke:none
    style G fill:#64748b,color:#fff,stroke:none

## Generics: The complaint that aged like milk

People love to complain about how Go didn't have generics. I get it -- back then, we were duplicating code or abusing `interface{}` like it was a religion. It was frustrating.

But that's over. [Go 1.18](https://tip.golang.org/doc/go1.18) brought generics. Not the "look how clever I am" kind of generics. The real-world, clean, no-BS kind.

I've built reusable, production-grade code with generics in Go, and it's boring in the best possible way. It doesn't add magic. It just saves time, cuts clutter, and lets you focus.

## Error Handling: You don't deserve *try-catch!*

People whine about Go's verbosity around error handling like it's some tragedy.

I actually respect it.

It makes you **look at your failure paths**. It doesn't let you pretend things are okay. There's no magic trapdoor. You deal with errors like an adult -- explicitly, predictably, and with clarity.

Now with `errors.Is`, `errors.As`, and `errors.Join`, it's not even painful anymore. It's direct. And if you're repeating `if err != nil` 10 times, that's not the language's fault -- that's your function decomposition screaming for help.

## Go does not babysit your Concurrency

Goroutines are simple. Channels are powerful. But Go doesn't stop you from screwing it up.

As the memes say, this isn't a bug -- it's a feature. Go gives you the raw tools to build concurrent systems. You want guarantees? Build them. You want race protection? Design them accordingly or use the built-in race detector. You want structured concurrency? There are patterns. Learn them.

Go assumes the person writing the code knows what they're doing. And I respect that.

> Java hides you behind abstractions. Go says, "Here's the knife. Don't cut yourself."

## Performance and Runtime: Quietly ruthless

Go's performance story doesn't make headlines, but it's been sharpening its edge for years.

I've deployed Go services that boot in under `50ms` and run for months without a hiccup. It doesn't brag. It just works. And when you actually care about reliability at scale, that's what matters.

## Go is not for everything

And it shouldn't be.

If you want total memory control, go write Rust. If you're obsessed with expressiveness and type wizardry, go enjoy Haskell. If you're building mobile apps, this ain't your tool either.

But if you're building modern backend systems -- the kind that run in the cloud, talk to queues, survive restarts, and serve real traffic -- Go is it.

**Java folks will try to throw Spring Boot into a Kubernetes cluster and call it modern. It's not. It's a legacy stack duct-taped into relevance.**

Go was built for this world. *Distributed. Scalable. Cloud-native.* Minimal by design. And if that feels limiting, maybe you're just too used to hiding behind abstraction soup.

graph LR
    A["Go"] --> B["Ship fast, simple"]
    C["Java / Spring"] --> D["Medium complexity"]
    E["Rust / Haskell"] --> F["Complex, more guarantees"]

    style A fill:#10b981,color:#fff,stroke:none
    style B fill:#10b981,color:#fff,stroke:none
    style C fill:#f59e0b,color:#fff,stroke:none
    style D fill:#f59e0b,color:#fff,stroke:none
    style E fill:#6366f1,color:#fff,stroke:none
    style F fill:#6366f1,color:#fff,stroke:none

## Personal take

I've built real systems in Go. Banking systems. Transaction processors. Cloud-native APIs. Stuff that handles money, user trust, and regulatory pressure.

It's not flashy. It's not fun in the academic sense. But Go keeps things simple and predictable and that's exactly what backend engineering demands.

If you want a language that lets you build boringly reliable systems at scale, Go's your bet.

Not because it's perfect. But because it **forces you to think clearly, fail loudly, and ship resiliently**.

And that's what good engineering is.


---

## How Goroutine Stacks Grow and Shrink

URL: https://ashwingopalsamy.in/writing/how-goroutine-stacks-grow-and-shrink/
Published: 2025-06-08
Tags: go-internals


If you ask any mid-senior Go dev what makes goroutines 'lightweight' and you'll get the standard reply:

> They start with **2 KB** of stack instead of 1 MB like OS threads.

They're not wrong. But they're not thinking deep enough.

Go's stack model isn't just a small preallocated buffer; it's **a live, evolving region of memory that resizes in realtime**, grows when needed, and (rarely) shrinks. It's also bounded. Not infinite. Bounded by hard design.

And *none of this* is your typical day-to-day developer concern. Until it is.

---

Let's start simple. Go's runtime gives each new goroutine **2 KB** of stack. That's tiny. But Go doesn't panic when you blow past it - it grows the stack dynamically, by allocating a new region (typically doubling the size) and copying the old stack frames over.

This is a silent, behind-the-scenes act of memory juggling that can happen **dozens or hundreds of times per process**, with no visibility unless you go looking.

Here's the kicker: **each goroutine has an upper stack limit** and it's not documented in bold in any official place.

### The hard upper bound per goroutine? Around 1 GB of stack.

Hit it, and the program **panics immediately**:

```
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow
```

That's not a soft fail. That's a crash.

And it's easier to hit than you think if you're writing recursive algorithms, parsing deeply nested data, or spawning goroutines in hot paths that grow quickly under concurrency.

---

### Stack Growth Lifecycle

Every time a goroutine's stack runs out of space, the runtime silently doubles the allocation and copies everything over. This is the full lifecycle from creation to regrowth.

graph LR
    A["New Goroutine\n2 KB stack"] --> B["Function\nCall"]
    B --> C{"Stack\nCheck"}
    C -->|Enough| D["Continue\nExecution"]
    C -->|Overflow| E["Allocate\n2x Stack"]
    E --> F["Copy Old\nFrames"]
    F --> G["Update\nPointers"]
    G --> D

    style A fill:#6366f1,color:#fff,stroke:none
    style C fill:#f59e0b,color:#fff,stroke:none
    style E fill:#10b981,color:#fff,stroke:none
    style F fill:#10b981,color:#fff,stroke:none
    style G fill:#10b981,color:#fff,stroke:none
    style D fill:#64748b,color:#fff,stroke:none
    style B fill:#64748b,color:#fff,stroke:none

---

Let's prove it.

Try this:

```go
package main

func deep(n int) {
    var buf [1024]byte // 1 KB per frame
    buf[0] = byte(n)
    if n > 0 {
        deep(n - 1)
    }
}

func main() {
    deep(4096 * 4096) // Push for 1 GB stack with 16 million calls
}
```

You'll crash. Every call uses 1 KB on the stack. 1M recursive calls = 1 GB.

```bash
> go run main.go

runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0x140201603a0 stack=[0x14020160000, 0x14040160000]
fatal error: stack overflow

runtime stack:
runtime.throw({0x104f9c923?, 0x100000000?})
... and more fluff ...
```

The stack doesn't shrink after this. It doesn't get reused by default. The runtime gives up.

---

Here's something most people don't know: **stack growth triggers memory copy operations.** Every time your goroutine blows past its stack limit, the runtime:

- Allocates a new larger stack
- Copies the existing stack to the new one
- Updates stack pointers and metadata
- Continues execution like nothing happened

This is *not free*. It introduces latency and can increase garbage collection overhead - because stacks contain pointers, and the Go GC must scan every live goroutine stack frame for reachable objects.

**The more your stacks grow, the more work your GC has to do.**

Even if those stacks are just frames, if they hold pointers, they're GC roots.

### Stack Size Progression

Stacks double on each growth event, from the initial 2 KB up to the ~1 GB ceiling. The GC can shrink stacks back down, but only under specific conditions.

graph TD
    S1["2 KB\n(initial)"] --> S2["4 KB"]
    S2 --> S3["8 KB"]
    S3 --> S4["16 KB"]
    S4 --> S5["32 KB"]
    S5 --> S6["..."]
    S6 --> S7["~1 GB\n(ceiling)"]

    S7 -.->|"GC shrink\n(if idle + mostly unused)"| S5
    S5 -.->|"GC shrink"| S3

    style S1 fill:#10b981,color:#fff,stroke:none
    style S2 fill:#10b981,color:#fff,stroke:none
    style S3 fill:#6366f1,color:#fff,stroke:none
    style S4 fill:#6366f1,color:#fff,stroke:none
    style S5 fill:#f59e0b,color:#fff,stroke:none
    style S6 fill:#f59e0b,color:#fff,stroke:none
    style S7 fill:#ef4444,color:#fff,stroke:none

---

A goroutine with a **2 KB** stack is cheap.

A goroutine that grows to **512 KB**, holds references to large objects, and lives long enough to survive multiple GC cycles? That's not cheap anymore. That's stealth memory overhead.

Let's look at this example:

```go
package main

import (
    "time"
)

func holdMemory(n int) {
    var data [128 * 1024]byte // 128 KB
    data[0] = 1
    time.Sleep(10 * time.Second) // Keep goroutine alive
}

func main() {
    for i := 0; i < 1000; i++ {
        go holdMemory(i)
    }
    time.Sleep(30 * time.Second)
}
```

You just spawned **1000** goroutines, each holding at least 128 KB on stack. That's **128 MB** of live stack memory **not counted in your heap**, but scanned by GC. And it only gets worse under load.

---

Now the part nobody talks about: **stack shrinking.**

Yes, Go does shrink goroutine stacks, **but only during garbage collection**, and only if:

- The goroutine is idle
- The stack is mostly unused
- The shrink won't cause immediate regrowth

In other words: *don't count on it.* Go is conservative with stack shrinkage. This means a burst of high-memory goroutines can bloat your memory profile **long after the work is done**, unless the GC kicks in and decides to do housecleaning - which it might not.

Want to observe it?

You can't. There is no public runtime metric for per-goroutine stack usage. You can't pprof it directly unless you attach custom logic. You can't even tell if a goroutine stack has been shrunk unless you look into a trace.

---

### Contiguous Stack Copy

When a stack outgrows its current allocation, Go allocates a new contiguous block at double the size, copies all frames, and adjusts every internal pointer. The old stack is then freed.

graph LR
    subgraph "Old Stack (4 KB)"
        O1["Frame 1"]
        O2["Frame 2"]
        O3["Pointer A\n→ Frame 1"]
    end

    subgraph "New Stack (8 KB)"
        N1["Frame 1\n(copied)"]
        N2["Frame 2\n(copied)"]
        N3["Pointer A'\n→ Frame 1\n(adjusted)"]
        N4["Free Space"]
    end

    O1 -->|copy| N1
    O2 -->|copy| N2
    O3 -->|"adjust + copy"| N3

    style O1 fill:#64748b,color:#fff,stroke:none
    style O2 fill:#64748b,color:#fff,stroke:none
    style O3 fill:#f59e0b,color:#fff,stroke:none
    style N1 fill:#6366f1,color:#fff,stroke:none
    style N2 fill:#6366f1,color:#fff,stroke:none
    style N3 fill:#10b981,color:#fff,stroke:none
    style N4 fill:none,stroke:#6366f1,stroke-dasharray:5

---

Want to go deeper? Try this:

```go
package main

import (
    "fmt"
    "runtime"
)

func recurse(n int) {
    var buf [1024]byte
    buf[0] = byte(n)
    if n > 0 {
        recurse(n - 1)
    }
}

func main() {
    var m runtime.MemStats
    runtime.ReadMemStats(&m)
    fmt.Println("Before:", m.StackInuse)

    recurse(1000)

    runtime.ReadMemStats(&m)
    fmt.Println("After:", m.StackInuse)
}
```

It prints stack usage in bytes before and after a deep recursion.

```bash
> go run main.go

Before: 262144
After: 294912
```

You'll see how the memory gets allocated, but never explicitly freed.

---

Let's hit one more unexplored angle: **stack growth can trigger GC pressure even without heap allocations.**

If you think your service has no memory leak because you aren't allocating on the heap, you're missing the point. A runaway stack holds pointers. Those pointers get scanned. That means GC is invoked **more often**, or **takes longer**, even if you aren't growing the heap.

This is how your **5ms** p99 turns into **100ms** not from bad code, but from unseen stack behavior.

---

There's no tuning knob for stack size.

No config.

No CLI flag to control initial stack size, max size, or shrink behavior.

The only way to manage it is **through code discipline**:

- Avoid recursive goroutines unless they terminate quickly
- Don't hold large structs or pointers deep in call graphs
- Be aware of implicit stack use via function calls
- Never assume goroutines are 'free'. Inspect their memory impact
- Use `GODEBUG=efence=1` to crash fast and find limits

---

You don't need to memorize internals. But you **do need to understand the consequences**.

Most devs won't talk about this. Few don't know either.

But now you do.

And if you write systems that scale, this will hit you eventually.

Better to learn it now while your stack's still small.


---

## Go Scheduler, Yield Points, and Infinite Loops

URL: https://ashwingopalsamy.in/writing/go-scheduler-yield-points-and-infinite-loops/
Published: 2025-05-24
Tags: go-internals


I've been reviewing some performance-critical code lately, and I keep coming back to this pattern:

```go
for {
    // tight polling loop
    if condition {
        break
    }
}
```

versus

```go
go func() {
    for {
        // background processing
    }
}()
```

On the surface, these look similar, both use infinite loops. But the runtime implications are fascinating, and I think we don't talk about this enough.

## The performance rabbit hole

Here's what got me thinking. I was optimizing a real-time data processor that needed sub-millisecond response times. The naive approach was throwing everything into goroutines:

```go
go func() {
    for {
        data := <-inputChan
        process(data)
        outputChan <- result
    }
}()
```

Standard Go idiom, right? But the scheduler overhead was killing us. Context switches, goroutine parking/unparking. All tiny costs that add up when you're processing millions of events per second.

The solution? Sometimes a plain `for {}` in the main goroutine actually performs better:

```go
func main() {
    for {
        select {
        case data := <-inputChan:
            process(data)
            outputChan <- result
        default:
            // yield briefly to scheduler
            runtime.Gosched()
        }
    }
}
```

No goroutine overhead. No scheduler interference. Just raw, tight loop performance when you need it.

## When Goroutines can become overhead

This isn't about goroutines being bad. They're one of Go's best features. But like any abstraction, they have costs. For most applications, those costs are negligible. For some applications, they matter.

I've seen codebases where developers reflexively wrap every loop in a goroutine because *"concurrency is good"*. But if you're not actually doing concurrent work, you're just adding overhead:

```go
// Unnecessary overhead
go func() {
    for i := 0; i < len(data); i++ {
        process(data[i])  // sequential work anyway
    }
}()

// Just do the work
for i := 0; i < len(data); i++ {
    process(data[i])
}
```

The goroutine version doesn't make this faster. It makes it slower. You've added scheduling overhead for no concurrent benefit.

## How the Go Scheduler Changes Everything

To understand why this choice matters, you need to know how Go's scheduler actually works. It's not just theory, this directly impacts your performance profile.

Go uses an **M:N** scheduler where **M** goroutines are multiplexed onto **N** OS threads. The key insight is that goroutines are cooperatively scheduled, not preemptively. They yield control at specific points:

- Channel operations
- System calls
- Memory allocator calls
- Explicit `runtime.Gosched()` calls
- Function calls (since Go 1.14, thanks to async preemption)

Here's the critical difference: a tight `for {}` loop without any of these yield points will monopolize its OS thread until the scheduler's async preemption kicks in *(roughly every 10ms)*.

Sometimes that's exactly what you want. An uninterrupted CPU time for hot loops.

But when you wrap that same loop in a goroutine, it competes with other goroutines for scheduling time. Each time the scheduler runs (which happens frequently), there's overhead:

```go
// This might get preempted constantly
go func() {
    for {
        // tight computation
        result := expensiveCalculation()
        if result > threshold {
            break
        }
    }
}()

// This runs uninterrupted until natural yield points
for {
    result := expensiveCalculation()
    if result > threshold {
        break
    }
}
```

The scheduler overhead includes context switching, stack management, and the coordination between the scheduler and your goroutines. For most code, this is negligible. For tight loops processing millions of operations, it's measurable.

## The GMP Model in Practice

Go's scheduler uses a **GMP** model: Goroutines (**G**) run on Machine threads (**M**) via Processors (**P**). Each **P** has a local run queue of goroutines, plus there's a global run queue. When you create a goroutine, it gets queued for scheduling.

graph TD
    subgraph "Global Run Queue"
        GQ["G5, G6, G7 ..."]
    end

    subgraph "P0 (Processor)"
        LQ0["Local Queue:\nG1, G2"]
        LQ0 --> M0["M0\n(OS Thread)"]
        M0 --> CPU0["CPU Core 0"]
    end

    subgraph "P1 (Processor)"
        LQ1["Local Queue:\nG3, G4"]
        LQ1 --> M1["M1\n(OS Thread)"]
        M1 --> CPU1["CPU Core 1"]
    end

    GQ -.->|"schedule"| LQ0
    GQ -.->|"schedule"| LQ1
    LQ0 -.->|"work steal"| LQ1

    style GQ fill:#64748b,color:#fff,stroke:none
    style LQ0 fill:#6366f1,color:#fff,stroke:none
    style LQ1 fill:#6366f1,color:#fff,stroke:none
    style M0 fill:#f59e0b,color:#fff,stroke:none
    style M1 fill:#f59e0b,color:#fff,stroke:none
    style CPU0 fill:#10b981,color:#fff,stroke:none
    style CPU1 fill:#10b981,color:#fff,stroke:none

The scheduler's work-stealing algorithm means goroutines can migrate between threads, which adds coordination overhead. For a single hot loop that doesn't need concurrency, this is pure cost with no benefit.

I've started thinking about this choice through the lens of scheduler pressure:

- **High-frequency tight loops**: Run on main goroutine to avoid scheduling overhead
- **Background/periodic work**: Use goroutines for natural yielding and fairness
- **I/O bound operations**: Definitely goroutines (blocking syscalls trigger scheduler naturally)
- **CPU-bound work that can be parallelized**: Multiple goroutines, but be mindful of coordination costs

### Cooperative Scheduling

Under cooperative scheduling, a goroutine holds the processor until it voluntarily yields at a known yield point, like a channel operation or a function call.

sequenceDiagram
    participant S as Scheduler
    participant A as Goroutine A
    participant B as Goroutine B

    S->>A: Schedule on P0
    activate A
    Note over A: Executing...
    A->>A: Channel send (yield point)
    A->>S: Yield control
    deactivate A
    S->>B: Schedule on P0
    activate B
    Note over B: Executing...
    B->>B: runtime.Gosched()
    B->>S: Yield control
    deactivate B
    S->>A: Re-schedule on P0
    activate A
    Note over A: Resumes execution
    deactivate A

### Async Preemption

But what about goroutines that never yield? Since Go 1.14, the `sysmon` background thread detects goroutines running longer than ~10ms and forces preemption via a SIGURG signal.

sequenceDiagram
    participant SM as sysmon
    participant G as Goroutine (tight loop)
    participant S as Scheduler
    participant G2 as Next Goroutine

    G->>G: Running for >10ms (no yield points)
    SM->>SM: Detects long-running G
    SM->>G: Send SIGURG
    Note over G: Signal handler fires,\nsaves state, marks preemptible
    G->>S: Suspended
    S->>G2: Schedule next goroutine
    activate G2
    Note over G2: Executing...
    deactivate G2
    S->>G: Re-schedule eventually

## Real-World Patterns

In practice, I see three main patterns where this distinction matters:

**1. Event loops in performance-critical paths:**

```go
// Main processing thread
for {
    select {
    case event := <-events:
        handleCriticalPath(event)
    case <-shutdown:
        return
    }
}
```

**2. Background workers:**

```go
// Background cleanup, metrics, etc.
go func() {
    ticker := time.NewTicker(time.Minute)
    defer ticker.Stop()
    for {
        select {
        case <-ticker.C:
            cleanup()
        case <-ctx.Done():
            return
        }
    }
}()
```

**3. Hybrid approaches:**

```go
// Main thread handles hot path
go backgroundWorker()  // Cold path in goroutine

for {
    select {
    case urgent := <-urgentChan:
        handleUrgent(urgent)  // Zero-copy, minimal overhead
    case routine := <-routineChan:
        routineWorkChan <- routine  // Delegate to background
    }
}
```

## Choosing the Right Pattern

The decision between a plain loop and a goroutine comes down to what you're optimizing for. This decision tree captures the key branching points.

graph LR
    A{"Need\nconcurrency?"}
    A -->|No| B["for {} loop\n(direct execution)"]
    A -->|Yes| C{"Bounded\nwork?"}
    C -->|Yes| D["Worker Pool\n(fixed goroutines)"]
    C -->|No| E{"I/O or\nCPU bound?"}
    E -->|I/O| F["Goroutine per task\n(scheduler handles blocking)"]
    E -->|CPU| G["select {} loop\n(controlled yielding)"]

    style A fill:#6366f1,color:#fff,stroke:none
    style B fill:#10b981,color:#fff,stroke:none
    style C fill:#f59e0b,color:#fff,stroke:none
    style D fill:#10b981,color:#fff,stroke:none
    style E fill:#f59e0b,color:#fff,stroke:none
    style F fill:#10b981,color:#fff,stroke:none
    style G fill:#10b981,color:#fff,stroke:none

## The nuance most don't talk about

The real insight isn't *"loops vs goroutines"*. It's understanding when the scheduler helps you and when it gets in your way. Most Go education focuses on the happy path where goroutines solve everything. But production systems often need more surgical approaches.

I've seen systems gain 30% throughput just by moving one critical loop out of a goroutine. I've also seen systems become unresponsive because someone removed goroutines that were providing necessary yielding points.

The trick is knowing which scenario you're in.

## When this actually matters?

To be clear, this level of optimization matters for maybe 5% of Go applications. If you're building typical web services, CRUD apps, or data pipelines, just use goroutines everywhere and call it a day. The scheduler overhead is negligible compared to I/O, database calls, and network latency.

But if you're **building at scale**, then these micro-optimizations can make the difference between meeting your SLAs and missing them.

## The takeaway

Go's runtime gives you both tools for a reason. Goroutines for most things, plain loops when you need maximum control. The art is recognizing which is which.

It's not about being clever. It's about understanding your performance profile and choosing the right abstraction for the job.


---

## Review Your Own PR First

URL: https://ashwingopalsamy.in/writing/review-your-own-pr-first/
Published: 2025-01-15
Tags: engineering-practices


Like most developers, I've been on both sides of Pull Requests (PRs) -- sending them out to colleagues and reviewing others' code in a sprint to meet deadlines. What I've noticed is that the best PRs, the ones that get merged smoothly and with minimal back-and-forth, are the ones where the author has already done a thorough self-review. It's not just about tidying up code style; it's about catching design flaws, typos, logic gaps and potential optimisation pitfalls before your teammates do.

In this post, I'd like to walk through why you should review your own PR first and provide some practical steps on how to do it. And hey, I'm writing this because I've personally seen the difference it makes while building internet-scale fintech systems for Europe, where a missed detail can lead to big headaches down the line.

---

### What Happens Without Self-Review

Here's the typical cycle when a PR goes out without self-review. The author pushes, the reviewer catches trivial issues, the author fixes, the reviewer re-reviews, finds more, and the cycle repeats. Three or more round trips before anything substantive gets discussed.

sequenceDiagram
    participant Author
    participant Reviewer

    Author->>Reviewer: Push PR for review
    Reviewer->>Author: 12 comments (typos, dead code, naming)
    Author->>Author: Fix trivial issues
    Author->>Reviewer: Push fixes, re-request review
    Reviewer->>Author: 5 more comments (missed edge case, style)
    Author->>Author: Fix again
    Author->>Reviewer: Push fixes, re-request review
    Reviewer->>Author: 2 more nits
    Note over Author,Reviewer: 3+ round trips before merge

---

## 1. Why Self-Review Is Critical

### a. Saves Time for Everyone

As soon as you open a Pull Request, your teammates set aside time to look at your code. If your PR is full of small mistakes -- like typos, unused variables (although unlikely to happen with languages like Go -- my primary and favorite programming language), or dead code -- reviewers will end up focusing on these easy catches instead of more architectural or design concerns. By cleaning up these obvious issues yourself, you let reviewers zero in on what really matters.

> Very early in my career, I once rushed a PR for a critical feature and ended up with a barrage of comments about redundant code and inconsistent naming. About 30 of them. The real logic flaw I introduced got overlooked for a while -- until it blew up in staging.
>
> Embarrassing? Yes. It taught me that even a quick self-review can spare everyone a lot of pain later.

### b. Improves Your Own Code Quality

Reading your own code diff is like stepping back to observe your painting from a distance. You notice patterns (or anti-patterns) that don't jump out when you're in the trenches writing code. Did you name that function well? Is there an awkward datastructure that might be simplified? These insights not only refine your current PR but also shape how you approach coding in the future.

### c. Builds Trust and Professionalism

Whether you're working in a small startup/scaleup (like where I do) or a large enterprise, your PR is a reflection of your work ethic. Taking time to polish your code before asking for a review signals respect for your colleagues' time. Over multiple sprints, that respect fosters trust and leads to more streamlined team communication.

Self-review compounds over time. Reviewers learn that your PRs are clean, so they focus on architecture and design instead of surface issues. This builds a feedback loop where reviews get shorter and more valuable with each sprint.

---

### What Happens With Self-Review

Contrast the earlier cycle with what happens when the author self-reviews before pushing. One round trip, one substantive comment, done.

sequenceDiagram
    participant Author
    participant Self as Self-Review
    participant Reviewer

    Author->>Self: Read own diff as reviewer
    Self->>Author: Catch 12 trivial issues locally
    Author->>Author: Fix naming, dead code, edge cases
    Author->>Author: Run tests, check linting
    Author->>Reviewer: Push clean PR for review
    Reviewer->>Author: 1 substantive design comment
    Author->>Author: Address feedback
    Author->>Reviewer: Push final fix
    Note over Author,Reviewer: 1 round trip to merge

---

## 2. How to Perform a Self-Review

### a. View the Diff As If You're Someone Else

I usually open GitHub (GitLab, or Bitbucket in your case), navigate to the **Files changed** tab, and read every line as though I'm the reviewer. Are the names clear? Is error handling consistent (which is a MUST with Go)? Is there test coverage for every new or changed piece of logic? (You can be conservative, but not here.) By shifting perspective, you'll catch issues you missed while coding.

> In a FinTech context, especially with microservices for accounts, transactions or credit cards, a single nil check might prevent a major production outage. Reading the diff as a reviewer helps me see if I've handled edge cases, like a missing value in a 3rd-party API response.

### b. Check Commit Messages and Branch Name

Yes, your PR title and description are important, but so are your commit messages. They form a living history of your project. Does it follow (try to) the [conventional-commit](https://gist.github.com/qoomon/5dfcdf8eec66a051ecd85625518cfd13) style? Does each commit represent a logical chunk of work? Or do you see "WIP: fix bug" repeated four times?

- **Atomic Commits**: Make sure each commit fixes or implements exactly one thing.
- **Good Commit Messages**: Follow a structure like `fix: handle missing field in transaction flow`, rather than "misc changes".

If you spot multiple fixes or refactors jammed into one commit, consider an interactive rebase to split them up. Similarly, ensure your **branch name** follows your team's convention (like `JIRA-5678-fix-transaction-timeout`) to keep things organized.

### c. Document Any Special Considerations

If there's something non-obvious about your approach -- for example, a tricky concurrency hack or a workaround for a known library bug -- mention it. Add inline comments in your code or elaborate in your PR description. Your Oncall Engineer will thank you while firefighting a production issue at 3 AM.

If your code references a known bug or limitation, link to any relevant tickets or documentation right in the PR or code comment. Clarity now saves major confusion later. Adding self-review comments on your own PR is also a great way to proactively explain tricky decisions.

### d. Validate Tests and Benchmarks

Every new or modified code path should ideally have a test -- unit, integration or end-to-end. Quickly run them locally (or rely on CI if it's robust enough) and check coverage reports if available. Did you add a new database migration script or an additional endpoint? Make sure you've tested for both success and failure scenarios.

> In a microservice handling accounts or transactions, a single missed test case might break the ledger for an entire day. Tests aren't just checkboxes; they're safety nets.

### e. Check for Style and Linting

Even small inconsistencies in code style can distract reviewers from more substantial issues. If your team uses linting tools or formatters like [golangci-lint](https://github.com/golangci/golangci-lint), [gofumpt](https://github.com/mvdan/gofumpt) or ESLint, run them before you open a PR. Fix any warnings or errors unless they're truly exceptions to your rule set.

---

### Self-Review Checklist

Before hitting "Request Review", walk through this checklist. Each "No" is a loop back to fix before pushing.

graph TD
    Start["Open your PR diff"] --> A{"Diff readable?\nClear naming?"}
    A -->|Yes| B{"Commit messages\nclear and atomic?"}
    A -->|No| A1["Fix naming,\nclean up diff"] --> A

    B -->|Yes| C{"Tests pass?\nCoverage adequate?"}
    B -->|No| B1["Rewrite commits,\ninteractive rebase"] --> B

    C -->|Yes| D{"Docs updated?\nPR description clear?"}
    C -->|No| C1["Add tests,\nfix failures"] --> C

    D -->|Yes| E{"Linting clean?\nNo warnings?"}
    D -->|No| D1["Update docs,\nadd PR context"] --> D

    E -->|Yes| F["Ready for review"]
    E -->|No| E1["Run linter,\nfix issues"] --> E

    style F fill:#10b981,color:#fff,stroke:none
    style Start fill:#64748b,color:#fff,stroke:none
    style A fill:#6366f1,color:#fff,stroke:none
    style B fill:#6366f1,color:#fff,stroke:none
    style C fill:#6366f1,color:#fff,stroke:none
    style D fill:#6366f1,color:#fff,stroke:none
    style E fill:#6366f1,color:#fff,stroke:none
    style A1 fill:#f59e0b,color:#fff,stroke:none
    style B1 fill:#f59e0b,color:#fff,stroke:none
    style C1 fill:#f59e0b,color:#fff,stroke:none
    style D1 fill:#f59e0b,color:#fff,stroke:none
    style E1 fill:#f59e0b,color:#fff,stroke:none

---

## 3. Common Pitfalls (And How to Avoid Them)

1. **Too Large PRs**
   - **Solution**: If you find your PR has grown too large, consider breaking it into smaller chunks. Maybe the database schema migration script can (in ideal cases -- SHOULD) be a separate PR.

2. **Neglecting Documentation**
   - **Solution**: If your changes include a new API endpoint or config file, update the README or relevant docs. Reviewing your own PR is a great time to spot missing documentation.

3. **Lack of Context in the PR Description**
   - **Solution**: Summarize what changed, why it changed, and any impacts on the system. This ensures the reviewers understand the context from the get-go.
   - **Another one:** I have a habit of adding PR comments pointing to my code explaining tricky bits or why I made a certain decision. It helps me think things through and makes it easier for reviewers to quickly grasp the rationale behind my decision later on.

4. **Forgetting to Rebase or Merge Main**
   - **Solution**: Before you finalize your PR, pull in the latest changes from `main` or `develop` (depending on your workflow). Fix any merge conflicts now rather than letting your reviewer handle them.

---

## 4. The Ripple Effect of a Good Self-Review

### a. Faster Approvals

If your PR is clean and well-structured, your reviewers won't have to spend time on trivial comments or guess your intentions. This leads to a more constructive review session where you can focus on potential design improvements and edge cases, ultimately speeding up merges.

### b. Better Team Morale

Pull requests often come with a bit of stress; nobody wants that dreaded "**Can you fix these 17 things?**" comment. A well-reviewed PR shows you respect the review process, which makes your teammates more eager to review your work. This mutual respect boosts morale and reduces the friction sometimes found in code review cycles.

### c. Stronger Codebase for Production-Grade Systems

In FinTech or any high-stakes industry, code reliability is paramount. Errors can be costly -- financially and reputationally. By catching small bugs and questionable logic early, you reduce the risk of these issues making their way to production.

> I remember we once traced a subtle floating-point rounding bug that only appeared in large batch transactions. Had I done a thorough self-review, I think I might've caught it. Instead, we found out during a production spike, leading to a hotfix scenario.

---

## 5. Final Thoughts

PR reviews are not just a box to check or to rely on others to correct our mistakes; they're a vital step in producing quality, maintainable software. By reviewing your own PR first -- treating it like someone else's code -- you'll create a better experience for both yourself and your reviewers. Think of it as a courtesy that doubles as a code-quality accelerator.

The simplest habit that pays the biggest dividends: before clicking "Request Review", open the Files changed tab and read every line as if a colleague wrote it. If something makes you pause, fix it. Your reviewers will notice the difference within one sprint.

Thanks for reading. If you found this helpful, feel free to leave a comment or share your own stories. I'd love to hear how self-review has impacted your codebase or your team's productivity.

Happy Coding -- and Happy Reviewing.


---

## The comparable Constraint in Go Generics

URL: https://ashwingopalsamy.in/writing/the-comparable-constraint-in-go-generics/
Published: 2024-12-25
Tags: go


While working on a generic function in Go, I once encountered this error: *'incomparable types in type set'*.

It led me to dig deeper into the `comparable` constraint - a seemingly straightforward feature that has profound implications for Go's generics. This wasn't my first brush with generics, but it highlighted an important nuance that's often overlooked. It's basic, yet surprisingly intuitive and an overlooked facet of Go generics that can save you from unnecessary debugging and potential runtime errors.

With this short post, I'll walk you through what `comparable` is, why it's useful and how you can leverage it for cleaner, type-safe Go code. If you're a beginner or even an early mid-level Gopher, this is for you.

---

## Why comparable?

Go's type system is famously simple and strict. But when generics entered the scene with Go 1.18, we gained new tools to write reusable, type-safe code. Alongside the introduction of `any` (an alias for `interface{}`), we got `comparable`. So, what's the deal?

Simply put, `comparable` is a constraint that ensures a type supports equality comparisons (`==` and `!=`).

In Go, only certain types are inherently comparable; these include primitive types like `int`, `float64` and `string`, but exclude types like slices, maps and functions. This is because slices and maps are reference types, meaning their equality cannot be determined by their contents but rather by their memory addresses. Functions, on the other hand, represent pointers to code blocks and are generally incomparable for practical purposes.

By enforcing the `comparable` constraint, Go helps ensure that generic functions relying on equality checks don't accidentally allow non-comparable types, preventing hard-to-debug runtime errors. Think of it as a guardrail for writing generic functions or types that rely on comparing values.

Let's say you want to write a generic function to check if a slice contains a specific value. Without `comparable`, you might inadvertently allow types like slices or maps, which are inherently not comparable in Go.

The result? A compiler error when you try to use `==` on those types.

With `comparable`, you can enforce at compile time that only valid, comparable types are used. This is incredibly valuable because it eliminates a whole class of runtime errors - like trying to compare slices or maps - before your code even runs. Compile-time checks provide immediate feedback, allowing you to fix issues early and ensuring that your functions behave predictably with the types they're designed to handle.

In a language like Go that emphasizes simplicity and reliability, this kind of safeguard aligns perfectly with the core philosophy of **letting the compiler do the hard work for you**.

Here's an example:

```go
type Array[T comparable] struct {
    data []T
}

func (a *Array[T]) Contains(value T) bool {
    for _, v := range a.data {
        if v == value {
            return true
        }
    }
    return false
}

func main() {
    arr := Array[int]{data: []int{1, 2, 3, 4, 5}}
    fmt.Println(arr.Contains(3))  // Output: true
    fmt.Println(arr.Contains(10)) // Output: false
}
```

Try using a slice or a map as the type parameter and the compiler will immediately stop you in your tracks. This guardrail is what makes `comparable` so valuable.

---

### Constraint Hierarchy

The relationship between `any`, `comparable`, and `constraints.Ordered` forms a clear hierarchy, where each level adds stronger guarantees about what operations a type supports.

graph TD
    A["any\n(all types)"]
    B["comparable\n(supports == and !=)"]
    C["constraints.Ordered\n(supports < > <= >=)"]

    A --> B
    B --> C

    A2["Examples: func, slice, map\nchan, interface"]
    B2["Examples: int, string, float64\nstruct, array, pointer"]
    C2["Examples: int, float64\nstring, ~int"]

    A --- A2
    B --- B2
    C --- C2

    style A fill:#64748b,color:#fff,stroke:none
    style B fill:#6366f1,color:#fff,stroke:none
    style C fill:#10b981,color:#fff,stroke:none
    style A2 fill:none,stroke:#64748b,stroke-dasharray:5
    style B2 fill:none,stroke:#6366f1,stroke-dasharray:5
    style C2 fill:none,stroke:#10b981,stroke-dasharray:5

---

### any vs. interface{}

Another player in the Go generics ecosystem is `any`, which is simply an alias for `interface{}`. Historically, `interface{}` has been a cornerstone of Go for representing any type, but its usage often confused newcomers due to its less intuitive name in the context of generics.

The introduction of `any` with Go 1.18 aimed to simplify this by providing a more semantic alias, making it immediately clear that the type accepts 'any' value. While functionally identical to `interface{}`, `any` better aligns with the intentions of generics and improves readability, especially in generic function signatures.

Consider this:

```go
func PrintValues[T any](values []T) {
    for _, v := range values {
        fmt.Println(v)
    }
}
```

Here, using `any` makes it clear that the function works with *any* type, without implying additional constraints.

Compare this to:

```go
func PrintValues[T interface{}](values []T) {
    // Same functionality, but less intuitive for generics
}
```

While `interface{}` still works, `any` feels more natural in the context of generics. It's a subtle shift, but one that makes your code more approachable, especially for newcomers.

---

### Real-World Scenarios: When to Use comparable and any

#### 1. Deduplication with comparable

Here's a quick example of using `comparable` to remove duplicates from a slice:

```go
func RemoveDuplicates[T comparable](input []T) []T {
    seen := make(map[T]bool)
    result := []T{}

    for _, v := range input {
        if !seen[v] {
            seen[v] = true
            result = append(result, v)
        }
    }
    return result
}

func main() {
    fmt.Println(RemoveDuplicates([]int{1, 2, 2, 3, 4, 4}))
    // Output: [1 2 3 4]

    fmt.Println(RemoveDuplicates([]string{"go", "go", "lang"}))
    // Output: [go lang]
}
```

#### 2. Flexible Utilities with any

Use `any` for functions that don't rely on constraints. For instance, a simple utility to print all elements in a slice:

```go
func PrintAll[T any](items []T) {
    for _, item := range items {
        fmt.Println(item)
    }
}

func main() {
    PrintAll([]int{1, 2, 3})
    PrintAll([]string{"hello", "world"})
}
```

No constraints are needed here and `any` signals that the function is open to all types.

---

## Common Pitfalls

#### 1. Trying to Compare Non-Comparable Types

If you've ever tried this:

```go
arr := Array[[]int]{data: [][]int{{1, 2}, {3, 4}}}
fmt.Println(arr.Contains([]int{1, 2})) // Compiler error
```

You'll get a compilation error because slices are not comparable. A workaround? Wrap your slices in a struct with a custom `Equals` method or use a hash function for comparison.

#### 2. Misusing any

Avoid overusing `any` when a constraint would make your code safer. For example, consider a function to find the maximum value in a slice:

```go
func FindMax[T any](items []T) T {
    max := items[0]
    for _, item := range items {
        if item > max {
            max = item
        }
    }
    return max
}
```

This code will fail to compile because `any` does not imply the ability to compare items using `>`. Instead, using `T comparable` ensures that the function can safely handle only types that support comparison:

```go
func FindMax[T comparable](items []T) T {
    max := items[0]
    for _, item := range items {
        if item > max {
            max = item
        }
    }
    return max
}
```

By adding the `comparable` constraint, you not only fix the compilation issue but also make the function's intent and requirements clear to anyone using it.

### Compile-Time vs Runtime: Why Constraints Matter

The real value of `comparable` is catching errors at compile time instead of letting them slip into production. Here's how the two paths compare:

graph LR
    subgraph "Using any (unsafe)"
        A1["func Contains\n[T any]"] --> B1["Pass []int\nas T"]
        B1 --> C1["Runtime Panic\n== on slice"]
    end

    subgraph "Using comparable (safe)"
        A2["func Contains\n[T comparable]"] --> B2["Pass []int\nas T"]
        B2 --> C2["Compile Error\n[]int not comparable"]
    end

    style C1 fill:#ef4444,color:#fff,stroke:none
    style C2 fill:#10b981,color:#fff,stroke:none
    style A1 fill:#64748b,color:#fff,stroke:none
    style A2 fill:#6366f1,color:#fff,stroke:none
    style B1 fill:#64748b,color:#fff,stroke:none
    style B2 fill:#6366f1,color:#fff,stroke:none

---

## Wrapping Up

Whether you're searching for a value, deduplicating a slice, or building your own generic utilities, understanding `comparable` can save you from unexpected bugs and runtime errors.

So, next time you find yourself scratching your head over an 'incomparable types' error, remember: it's not you - it's Go, nudging you toward better design. And if you're still unsure? That's fine too. Learning the quirks of a language is part of the fun (and frustration) of being a developer.


---

## What Happens Before main() in Go

URL: https://ashwingopalsamy.in/writing/what-happens-before-main-in-go/
Published: 2024-12-13
Tags: go-internals


When we first start with Go, the `main` function seems almost too simple. A single entry point, a straightforward `go run main.go` and voila -- our program is up and running.

But as we dig deeper, there's a subtle, well-thought-out process churning behind the curtain. Before `main` even begins, the Go runtime orchestrates a careful initialization of all imported packages, runs their `init` functions and ensures that everything's in the right order -- no messy surprises allowed.

The way Go arranges this has neat details to it that every Go developer should be aware of, as this influences how we structure our code, handle shared resources and even communicate errors back to the system.

Let's explore some common scenarios and questions that highlight what's really going on before and after `main` kicks into gear.

---

### Program Startup Sequence

Before your code runs a single line, the Go runtime has already done significant work. Here's the full sequence from binary execution to your first `fmt.Println`.

sequenceDiagram
    participant OS
    participant Runtime
    participant Packages
    participant main

    OS->>Runtime: Execute binary
    Runtime->>Runtime: Bootstrap (scheduler, GC, memory)
    Runtime->>Packages: Init packages in dependency order
    Packages->>Packages: Run init() functions
    Packages-->>Runtime: All packages initialized
    Runtime->>main: Call main()
    main->>main: Main goroutine runs
    main-->>OS: os.Exit or return

---

## Before main: Orderly Initialization and the Role of init

Picture this: you've got multiple packages, each with their own `init` functions. Maybe one of them configures a database connection, another sets up some logging defaults and a third initializes a lambda worker, with a fourth initializing an SQS queue listener.

By the time `main` runs, you want everything ready -- no half-initialized states or last-minute surprises.

**Example: Multiple Packages and init Order**

```go
// db.go
package db

import "fmt"

func init() {
    fmt.Println("db: connecting to the database...")
    // Imagine a real connection here
}

// cache.go
package cache

import "fmt"

func init() {
    fmt.Println("cache: warming up the cache...")
    // Imagine setting up a cache here
}

// main.go
package main

import (
    _ "app/db"   // blank import for side effects
    _ "app/cache"
    "fmt"
)

func main() {
    fmt.Println("main: starting main logic now!")
}
```

When you run this program, you'll see:

```
db: connecting to the database...
cache: warming up the cache...
main: starting main logic now!
```

The database initializes first (since `main` imports `db`), then the cache and finally `main` prints its message. Go guarantees that all imported packages are initialized before `main` runs. This dependency-driven order is key. If `cache` depended on `db`, you'd be sure `db` finished its setup before `cache`'s `init` ran.

Go initializes packages in **dependency order**, not import order. If `cache` imports `db`, then `db` will always initialize first regardless of how the imports are listed in `main.go`.

### Ensuring a Specific Initialization Order

Now, what if you absolutely need `db` initialized before `cache`, or vice versa?

The natural approach is to ensure `cache` depends on `db` or is imported after `db` in `main`. Go initializes packages in the order of their dependencies, not the order of imports listed in `main.go`.

A trick that we use is a blank import: `_ "path/to/package"` -- to force initialization of a particular package. But I wouldn't rely on blank imports as a primary method; it can make dependencies less clear and lead to maintenance headaches.

Instead, consider structuring packages so their initialization order emerges naturally from their dependencies. If that's not possible, maybe the initialization logic shouldn't rely on strict sequencing at compile time. You could, for instance, have `cache` check if `db` is ready at runtime, using a `sync.Once` or a similar pattern.

### Package Dependency Ordering

Here's how Go resolves initialization order across a dependency graph. The key insight: leaf packages (those with no further imports) initialize first, and it works its way up to `main`.

graph TD
    main["main\n(init last)"]
    A["Package A"]
    B["Package B"]
    C["Package C"]

    main --> A
    main --> B
    A --> C

    style C fill:#10b981,color:#fff,stroke:none
    style A fill:#10b981,color:#fff,stroke:none
    style B fill:#6366f1,color:#fff,stroke:none
    style main fill:#64748b,color:#fff,stroke:none

**Init order: C -> A -> B -> main.** C has no dependencies, so it initializes first. A depends on C, so it goes next. B has no unmet dependencies at that point. main is always last.

---

### Avoiding Circular Dependencies

Circular dependencies at the initialization level are a big no-no in Go. If package **A** imports **B** and **B** tries to import **A**, you've just created a circular dependency.

Go will refuse to compile, saving you from a world of confusing runtime issues. This might feel strict, but trust me, it's better to find these problems early rather than debugging weird initialization states at runtime.

---

### Dealing with Shared Resources and sync.Once

Imagine a scenario where packages **A** and **B** both depend on a shared resource -- maybe a configuration file or a global settings object. Both have `init` functions and both try to initialize that resource. How do you ensure the resource is only initialized once?

A common solution is to place the shared resource initialization behind a `sync.Once` call. This ensures that the initialization code runs exactly one time, even if multiple packages trigger it.

**Example: Ensuring Single Initialization**

```go
// config.go
package config

import (
    "fmt"
    "sync"
)

var (
    once      sync.Once
    someValue string
)

func init() {
    once.Do(func() {
        fmt.Println("config: initializing shared resource...")
        someValue = "initialized"
    })
}

func Value() string {
    return someValue
}
```

Now, no matter how many packages import `config`, the initialization of `someValue` happens only once. If package A and B both rely on `config.Value()`, they'll both see a properly initialized value.

Use `sync.Once` when you need lazy or guarded initialization of shared resources. It's safer than relying on `init()` ordering for cross-package resource setup.

### Choosing Between init and sync.Once

Not sure which to reach for? Here's a quick decision guide.

graph LR
    Q{"What kind of\ninitialization?"}
    A["Must happen\nat import time"]
    B["Can be deferred\n(lazy init)"]
    C["Need both"]

    R1["Use init()"]
    R2["Use sync.Once"]
    R3["init() for must-have\nsync.Once for expensive/optional"]

    Q --> A --> R1
    Q --> B --> R2
    Q --> C --> R3

    style R1 fill:#10b981,color:#fff,stroke:none
    style R2 fill:#6366f1,color:#fff,stroke:none
    style R3 fill:#f59e0b,color:#fff,stroke:none
    style Q fill:#64748b,color:#fff,stroke:none
    style A fill:#64748b,color:#fff,stroke:none
    style B fill:#64748b,color:#fff,stroke:none
    style C fill:#64748b,color:#fff,stroke:none

---

### Multiple init Functions in a Single File or Package

You can have multiple `init` functions in the same file and they'll run in the order they appear. Across multiple files in the same package, Go runs `init` functions in a consistent, but not strictly defined order. The compiler might process files in alphabetical order, but you shouldn't rely on that. If your code depends on a specific sequence of `init` functions within the same package, that's often a sign to refactor. Keep `init` logic minimal and avoid tight coupling.

**Legitimate Uses vs. Anti-Patterns**

`init` functions are best used for simple setup: registering database drivers, initializing command-line flags or setting up a logger. Complex logic, long-running I/O or code that might panic without good reason are better handled elsewhere.

> As a rule of thumb, if you find yourself writing a lot of logic in `init`, you might consider making that logic explicit in `main`.

---

### Exiting with Grace and Understanding os.Exit()

Go's `main` doesn't return a value. If you want to signal an error to the outside world, `os.Exit()` is your friend. But keep in mind: calling `os.Exit()` terminates the program immediately.

`os.Exit()` terminates the program immediately. **No deferred functions run**, no `panic` stack traces print. Always perform cleanup explicitly before calling `os.Exit()`.

**Example: Cleanup Before Exit**

```go
package main

import (
    "fmt"
    "os"
)

func main() {
    if err := doSomethingRisky(); err != nil {
        fmt.Println("Error occurred, performing cleanup...")
        cleanup()       // Make sure to clean up before calling os.Exit
        os.Exit(1)
    }
    fmt.Println("Everything went fine!")
}

func doSomethingRisky() error {
    // Pretend something failed
    return fmt.Errorf("something bad happened")
}

func cleanup() {
    // Close files, databases, flush buffers, etc.
    fmt.Println("Cleanup done!")
}
```

If you skip the cleanup call and jump straight to `os.Exit(1)`, you lose the chance to clean up resources gracefully.

#### Other Ways to End a Program

You can also end a program through a `panic`. A `panic` that's not recovered by `recover()` in a deferred function will crash the program and print a stack trace. This is handy for debugging but not ideal for normal error signaling. Unlike `os.Exit()`, a `panic` gives deferred functions a chance to run before the program ends, which can help with cleanup, but it also might look less tidy to end-users or scripts expecting a clean exit code.

Signals (like `SIGINT` from Cmd+C) can also terminate the program. If you're thorough, you can catch signals and handle them gracefully.

---

## Runtime, Concurrency and the main Goroutine

Initialization happens before any goroutines are launched, ensuring no race conditions at startup. Once `main` begins, however, you can spin up as many goroutines as you like.

> **It's important to note that the `main` function itself runs in a special "main goroutine" started by the Go runtime. If `main` returns, the entire program exits -- even if other goroutines are still doing work.**

This is a common gotcha: just because you started background goroutines doesn't mean they keep the program alive. Once main finishes, everything shuts down.

```go
package main

import (
    "fmt"
    "time"
)

func main() {
    go func() {
        time.Sleep(2 * time.Second)
        fmt.Println("Goroutine finished its job!")
    }()

    // If we simply return here, the main goroutine finishes,
    // and the program exits immediately, never printing the above message.

    time.Sleep(3 * time.Second)
    fmt.Println("Main is done, exiting now!")
}
```

In this example, the goroutine prints its message only because `main` waits 3 seconds before ending. If `main` ended sooner, the program would terminate before the goroutine completed. The runtime doesn't "wait around" for other goroutines when `main` exits.

If your logic demands waiting for certain tasks to complete, consider using synchronization primitives like `WaitGroup` or channels to signal when background work is done.

### What if a Panic Occurs During Initialization?

If a panic happens during `init`, the whole program terminates. No `main`, no recovery opportunity. You'll see a panic message that can help you debug. This is one reason to keep `init` functions simple, predictable and free of complex logic that might blow up unexpectedly.

---

## Wrapping It Up

By the time `main` runs, Go has already done a ton of invisible legwork: it's initialized all your packages, run every `init` function and checked that there are no nasty circular dependencies lurking around. Understanding this process gives you more control and confidence in your application's startup sequence.

When something goes wrong, you know how to exit cleanly and what happens to deferred functions. When your code grows more complex, you know how to enforce initialization order without resorting to hacks. And if concurrency comes into play, you know that the race conditions start after `init` runs, not before.

These little insights make Go's seemingly simple `main` function feel like the tip of an iceberg. If you have your own tricks, pitfalls you've stumbled into, or questions about these internals, I'd love to hear them.

After all, we're all still learning -- and that's half the fun of being a Go developer.


---

## Go Pointers and Memory Management

URL: https://ashwingopalsamy.in/writing/go-pointers-and-memory-management/
Published: 2024-11-17
Tags: go-internals


When I first started learning Go, I was intrigued by its approach to memory management, especially when it came to pointers. Go handles memory in a way that's both efficient and safe, but it can be a bit of a black box if you don't peek under the hood a bit.

With this post, I want to share some insights into how Go manages memory with pointers, the ***stack*** and ***heap***, and concepts like 'escape analysis' and 'garbage collection'. Along the way, we'll look at code samples that illustrate these in practice.

## Understanding Stack and Heap Memory

Before diving into pointers in Go, it's helpful to understand how the stack and heap work. These are two areas of memory where variables can be stored, each with its own characteristics.

- **Stack**: This is a region of memory that operates in a last-in, first-out manner. It's fast and efficient, used for storing variables with short-lived scope, like local variables within functions.

- **Heap**: This is a larger pool of memory used for variables that need to live beyond the scope of a function, such as data that's returned from a function and used elsewhere.

In Go, the compiler decides whether to allocate variables on the stack or the heap based on how they're used. This decision-making process is called **escape analysis**, which we'll explore in more detail later.

graph TD
    subgraph STACK["Goroutine Stack"]
        S1["Local variables"]
        S2["Function parameters"]
        S3["Return addresses"]
    end

    subgraph HEAP["Shared Heap"]
        H1["Escaped variables"]
        H2["Pointers returned<br/>from functions"]
        H3["Closure captures"]
    end

    EA{{"Escape Analysis<br/>(compile time)"}}

    EA -- "does not escape" --> STACK
    EA -- "escapes to heap" --> HEAP

    style STACK fill:#10b981,color:#fff,stroke:none
    style HEAP fill:#6366f1,color:#fff,stroke:none
    style EA fill:#f59e0b,color:#fff,stroke:none
    style S1 fill:#10b981,color:#fff,stroke:none
    style S2 fill:#10b981,color:#fff,stroke:none
    style S3 fill:#10b981,color:#fff,stroke:none
    style H1 fill:#6366f1,color:#fff,stroke:none
    style H2 fill:#6366f1,color:#fff,stroke:none
    style H3 fill:#6366f1,color:#fff,stroke:none

## Passing by Value: The Default Behavior

In Go, when you pass variables like integer, string, or boolean to a function, they are naturally passed by value. This means a copy of the variable is made, and the function works with that copy. This means, any change made to the variable inside the function will not affect the variable outside its scope.

Here's a simple example:

```go
package main

import "fmt"

func increment(num int) {
    num++
    fmt.Printf("Inside increment(): num = %d, address = %p \n", num, &num)
}

func main() {
    n := 21
    fmt.Printf("Before increment(): n = %d, address = %p \n", n, &n)
    increment(n)
    fmt.Printf("After increment(): n = %d, address = %p \n", n, &n)
}
```

**Output:**

```text
Before increment(): n = 21, address = 0xc000012070
Inside increment(): num = 22, address = 0xc000012078
After increment(): n = 21, address = 0xc000012070
```

In this code:

- The `increment()` function receives a copy of `n`.
- The addresses of `n` in `main()` and `num` in `increment()` are different.
- Modifying `num` inside `increment()` doesn't affect `n` in `main()`.

**Takeaway**: Passing by value is safe and straightforward, but for large data structures, copying may become inefficient.

## Introducing Pointers: Passing by Reference

To modify the original variable inside a function, you can pass a pointer to it. A pointer holds the memory address of a variable, allowing functions to access and modify the original data.

Here's how you can use pointers:

```go
package main

import "fmt"

func incrementPointer(num *int) {
    (*num)++
    fmt.Printf("Inside incrementPointer(): num = %d, address = %p \n", *num, num)
}

func main() {
    n := 42
    fmt.Printf("Before incrementPointer(): n = %d, address = %p \n", n, &n)
    incrementPointer(&n)
    fmt.Printf("After incrementPointer(): n = %d, address = %p \n", n, &n)
}
```

**Output:**

```text
Before incrementPointer(): n = 42, address = 0xc00009a040
Inside incrementPointer(): num = 43, address = 0xc00009a040
After incrementPointer(): n = 43, address = 0xc00009a040
```

In this example:

- We pass the address of `n` to `incrementPointer()`.
- Both `main()` and `incrementPointer()` refer to the same memory address.
- Modifying `num` inside `incrementPointer()` affects `n` in `main()`.

**Takeaway**: Using pointers allows functions to modify the original variable, but it introduces considerations about memory allocation.

## Memory Allocation with Pointers

When you create a pointer to a variable, Go needs to ensure that the variable lives as long as the pointer does. This often means allocating the variable on the ***heap*** rather than the ***stack***.

Consider this function:

```go
func createPointer() *int {
    num := 100
    return &num
}
```

Here, `num` is a local variable within `createPointer()`. If `num` were stored on the stack, it would be cleaned up once the function returns, leaving a dangling pointer. To prevent this, Go allocates `num` on the heap so that it remains valid after `createPointer()` exits.

Go **never** produces dangling pointers. Unlike C/C++, where returning a pointer to a local variable is undefined behavior, Go's escape analysis detects this at compile time and promotes the variable to the heap. The garbage collector then ensures the memory stays alive as long as any pointer references it. This is one of Go's strongest safety guarantees -- you get pointer semantics without manual memory management.

**Dangling Pointers**

A **dangling pointer** occurs when a pointer refers to memory that has already been freed.

Go prevents dangling pointers with its garbage collector, ensuring that memory is not freed while it is still referenced. However, holding onto pointers longer than necessary can lead to increased memory usage or memory leaks in certain scenarios.

## Escape Analysis: Deciding Stack vs. Heap Allocation

Escape analysis determines whether variables need to live beyond their function scope. If a variable is returned, stored in a pointer, or captured by a goroutine, it escapes and is allocated on the heap. However, even if a variable doesn't escape, the compiler might allocate it on the heap for other reasons, such as optimization decisions or stack size limitations.

flowchart TD
    A["Variable declared"] --> B{"Returned as<br/>pointer?"}
    B -- Yes --> HEAP["Allocate on Heap"]
    B -- No --> C{"Captured by<br/>closure?"}
    C -- Yes --> HEAP
    C -- No --> D{"Assigned to<br/>interface?"}
    D -- Yes --> HEAP
    D -- No --> E{"Too large<br/>for stack?"}
    E -- Yes --> HEAP
    E -- No --> STACK["Allocate on Stack"]

    style A fill:#64748b,color:#fff,stroke:none
    style B fill:#64748b,color:#fff,stroke:none
    style C fill:#64748b,color:#fff,stroke:none
    style D fill:#64748b,color:#fff,stroke:none
    style E fill:#64748b,color:#fff,stroke:none
    style HEAP fill:#ef4444,color:#fff,stroke:none
    style STACK fill:#10b981,color:#fff,stroke:none

**Example of a Variable Escaping:**

```go
package main

import "fmt"

func createSlice() []int {
    data := []int{1, 2, 3}
    return data
}

func main() {
    nums := createSlice()
    fmt.Printf("nums: %v\n", nums)
}
```

In this code:

- The slice `data` in `createSlice()` escapes because it's returned and used in `main()`.
- The underlying array of the slice is allocated on the **heap**.

**Understanding Escape Analysis with** `go build -gcflags '-m'`

You can see what Go's compiler decides by using the `-gcflags '-m'` option:

```bash
go build -gcflags '-m' main.go
```

This will output messages indicating whether variables escape to the heap.

Run `go build -gcflags '-m'` on any Go file to see the compiler's escape analysis decisions. Adding a second `-m` (`-gcflags '-m -m'`) gives even more detailed reasoning. This is one of the most underused profiling tools in Go -- it tells you exactly which allocations hit the heap and why, without needing to run a full benchmark.

## Garbage Collection in Go

Go uses a garbage collector to manage memory allocation and deallocation on the heap. It automatically frees memory that's no longer referenced, helping prevent memory leaks.

**Example:**

```go
package main

import "fmt"

type Node struct {
    Value int
    Next  *Node
}

func createLinkedList(n int) *Node {
    var head *Node
    for i := 0; i < n; i++ {
        head = &Node{Value: i, Next: head}
    }
    return head
}

func main() {
    list := createLinkedList(1000000)
    fmt.Println("Linked list created")
    // The garbage collector will clean up when 'list' as it was not used
}
```

In this code:

- We create a linked list with 1,000,000 nodes.
- Each `Node` is allocated on the heap because it escapes the scope of `createLinkedList()`.
- The garbage collector frees the memory when the list is no longer needed.

**Takeaway**: Go's garbage collector simplifies memory management but at times, may introduce overhead.

## Potential Pitfalls with Pointers

While pointers are powerful, they can lead to issues if not used carefully.

### Dangling Pointers (Continued)

Although Go's garbage collector helps prevent dangling pointers, you can still run into problems if you hold onto pointers longer than necessary.

**Example:**

```go
package main

import (
    "fmt"
    "time"
)

func main() {
    data := createData()
    fmt.Println("Data created")
    time.Sleep(10 * time.Second)
    fmt.Println("Data still in use:", data[0]) // this pointer is not dereferenced yet
}

func createData() *[]int {
    data := make([]int, 1000000)
    return &data
}
```

In this code:

- `data` is a large slice allocated on the heap.
- By keeping a reference to it (`[]int`), we prevent the garbage collector from freeing the memory.
- This can lead to increased memory usage if not managed properly.

### Concurrency Issues - Data Race with Pointers

Data races are one of the most insidious bugs in concurrent Go programs. They produce non-deterministic results, are difficult to reproduce, and can silently corrupt data. Always run `go test -race` to detect them. The race detector catches most races at runtime, but it only finds races in code paths that actually execute during the test.

Here's an example where pointers are directly involved:

```go
package main

import (
    "fmt"
    "sync"
)

func main() {
    var wg sync.WaitGroup
    counter := 0
    counterPtr := &counter // Pointer to the counter

    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            *counterPtr++ // Dereference the pointer and increment
            wg.Done()
        }()
    }

    wg.Wait()
    fmt.Println("Counter:", *counterPtr)
}
```

**Why This Code Fails:**

- Multiple goroutines dereference and increment the pointer `counterPtr` without any synchronization.
- This leads to a data race because multiple goroutines access and modify the same memory location concurrently without synchronization. The operation `*counterPtr++` involves multiple steps (read, increment, write) and is not thread-safe.

sequenceDiagram
    participant G1 as Goroutine 1
    participant M as *counter (memory)
    participant G2 as Goroutine 2

    Note over G1,G2: Without sync - Data Race
    G1->>M: Read *counter = 0
    G2->>M: Read *counter = 0
    G1->>M: Write *counter = 1
    G2->>M: Write *counter = 1
    Note over M: Expected 2, got 1

    Note over G1,G2: With sync.Mutex - Safe
    G1->>M: Lock()
    G1->>M: Read *counter = 0
    G1->>M: Write *counter = 1
    G1->>M: Unlock()
    G2->>M: Lock()
    G2->>M: Read *counter = 1
    G2->>M: Write *counter = 2
    G2->>M: Unlock()
    Note over M: Correct: 2

**Fixing the Data Race:**

We can fix this by adding synchronization with a mutex:

```go
package main

import (
    "fmt"
    "sync"
)

func main() {
    var wg sync.WaitGroup
    var mu sync.Mutex
    counter := 0
    counterPtr := &counter // Pointer to the counter

    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            mu.Lock()
            *counterPtr++ // Safely dereference and increment
            mu.Unlock()
            wg.Done()
        }()
    }

    wg.Wait()
    fmt.Println("Counter:", *counterPtr)
}
```

**How This Fix Works:**

- The `mu.Lock()` and `mu.Unlock()` ensure that only one goroutine accesses and modifies the pointer at a time.
- This prevents race conditions and ensures the final value of `counter` is correct.

---

## What does Go's Language Specification say?

It's worth noting that **Go's language specification doesn't directly dictate whether variables are allocated on the stack or the heap.** These are runtime and compiler implementation details, allowing for flexibility and optimizations that can vary across Go versions or implementations.

This means:

- The way memory is managed can change between different versions of Go.
- You shouldn't rely on variables being allocated in a specific area of memory.
- Focus on writing clear and correct code rather than trying to control memory allocation.

**Example:**

Even if you expect a variable to be allocated on the stack, the compiler might decide to move it to the heap based on its analysis.

```go
package main

func main() {
    var data [1000]int
    // The compiler may choose to allocate 'data' on the heap
    // if it deems it more efficient
}
```

**Takeaway**: As the memory allocation details are internal implementation and not part of the Go Language Specification, these information are only general guidelines and not fixed rules which might change at a later date.

## Balancing Performance and Memory Usage

When deciding between passing by value or by pointer, we must consider the size of the data and the performance implications.

**Passing Large Structs by Value:**

```go
type LargeStruct struct {
    Data [10000]int
}

func processValue(ls LargeStruct) {
    // Processing data
}

func main() {
    var ls LargeStruct
    processValue(ls) // Copies the entire struct
}
```

**Passing Large Structs by Pointer:**

```go
func processPointer(ls *LargeStruct) {
    // Processing data
}

func main() {
    var ls LargeStruct
    processPointer(&ls) // Passes a pointer, avoids copying
}
```

**Considerations:**

- Passing by value is safe and straightforward but can be inefficient for large data structures.
- Passing by pointer avoids copying but requires careful handling to avoid concurrency issues.

## From the field experience

In early career, I recall a time when I was optimizing a Go application that processed large sets of data. Initially, I passed large structs by value, assuming it would simplify reasoning about the code. However, I happened to notice comparably high memory usage and frequent garbage collection pauses.

After profiling the application using Go's `pprof` tool in a pair programming with my senior, we found that copying large structs was a bottleneck. We refactored the code to pass pointers instead of values. This reduced memory usage and improved performance significantly.

But the change wasn't without challenges. We had to ensure that our code was thread-safe since multiple goroutines were now accessing shared data. We implemented synchronization using mutexes and carefully reviewed the code for potential race conditions.

**Lesson Learned**: Very early understanding how Go handles memory allocation can help you write more efficient code, as it's essential to balance performance gains with code safety and maintainability.

## Final Thoughts

Go's approach to memory management (like how it does everywhere else) strikes a balance between performance and simplicity. By abstracting away many low-level details, it allows developers to focus on building robust applications without getting bogged down in manual memory management.

Key points to remember:

- **Passing by value** is simple but can be inefficient for large data structures.
- **Using pointers** can improve performance but requires careful handling to avoid issues like data races.
- **Escape analysis** determines whether variables are allocated on the stack or heap, but this is an internal detail.
- **Garbage collection** helps prevent memory leaks but might introduce overhead.
- **Concurrency** requires synchronization when shared data is involved.

By keeping these concepts in mind and using Go's tools to profile and analyze your code, you can write efficient and safe applications.


---

## Git Practices for Production Codebases

URL: https://ashwingopalsamy.in/writing/git-practices-for-production-codebases/
Published: 2024-11-14
Tags: engineering-practices


Git is the backbone of every software project. Whether you're squashing a bug, developing a feature, or tracing a production issue, Git quietly keeps track of everything. But let's face it -- Git can be as much of a headache as it is a lifesaver, especially if the history is messy or branches are all over the place.

Working in **highly critical, production-grade FinTech systems**, I've learned that Git isn't just a tool -- it's a shared language and a safety net. In environments where even minor mistakes can ripple across services, impacting compliance, customers, and trust, clean Git workflows become non-negotiable. Having managed **core banking systems** within this context, I always strive for structured commits, well-defined branches, and clear pull requests (PRs) to maintain the integrity of the codebase.

In FinTech and regulated environments, every change must be auditable and traceable. Clean Git practices are not just about developer productivity -- they are a compliance requirement. Structured commit histories, well-scoped branches, and documented PRs form the paper trail that auditors and incident responders rely on.

Here's a guide to **Git best practices** I almost follow (and aspire to), aimed at keeping repositories clean, collaborative, and resilient in the face of production challenges.

---

## Commits: The Backbone of Your Codebase

A Git history should feel like a well-documented timeline of your project's development, not a chaotic log of random changes. In highly critical environments, where changes must be audited and traceable, well-structured commits are crucial.

### 1. Write Atomic Commits

An atomic commit focuses on one thing -- fixing a bug, adding a feature, or refactoring code. This ensures every commit is clear, self-contained, and easy to understand. It also makes debugging and rollbacks safer.

**Example:**

- **Good:**

```bash
feat: add endpoint for retrieving user account balances
fix: resolve timeout issue in interest calculation
```

- **Bad:**

```bash
misc: fix bugs and add features
```

In early career, I've learned *(the hard way)* that bundling unrelated changes into a single commit creates confusion and risks during rollbacks, especially when a quick fix is required for production.

---

### 2. Use Descriptive Commit Messages

Your commit message should explain **what changed** and, if needed, **why**. Following a consistent format helps everyone on the team (including your future self) understand what's going on.

```text
<type>(<scope>): <subject>
<BLANK LINE>
<body (optional)>
```

**Examples:**

- `fix(auth): resolve token expiration handling for API calls`
- `feat(worker): implement batch processing for interest accrual`

These messages don't just help during reviews -- they're lifesavers when digging through logs or debugging an issue six months down the line.

The [Conventional Commits Specification](https://www.conventionalcommits.org/) formalizes this format. It pairs well with tools like **commitlint** and enables automatic changelog generation, semantic versioning, and structured release notes from your commit history alone.

graph LR
    A["feat(parser): add bitmap validation"]

    A --> B["<b>type</b><br/>feat"]
    A --> C["<b>scope</b><br/>parser"]
    A --> D["<b>description</b><br/>add bitmap validation"]

    style A fill:#64748b,color:#fff,stroke:none
    style B fill:#10b981,color:#fff,stroke:none
    style C fill:#6366f1,color:#fff,stroke:none
    style D fill:#f59e0b,color:#fff,stroke:none

---

### 3. Automate Commit Linting

No matter how disciplined we are, it's easy to slip up. That's where **Commitlint** comes in. It's a lightweight tool that ensures your commit messages follow a defined convention, like **Conventional Commits**.

**Tools for Commitlint:**

1. **[Commitlint by Conventional-Changelog](https://github.com/conventional-changelog/commitlint):**
   This is one of the most popular Commitlint tools. It's simple, extensible, and works seamlessly with Husky to enforce commit message rules during pre-commit or pre-push hooks.

2. **[Commitlint by ConventionalCommit](https://github.com/conventionalcommit/commitlint):**
   Written in **Golang**, this lightweight Commitlint tool is fast and easy to set up for smaller teams or those already using Go. It's perfect if you prefer tools that feel native to your tech stack. **My personal favorite**.

3. **[Conventional Commits Specification](https://www.conventionalcommits.org/)**
   A useful guide to the conventions enforced by these tools.

Setting up Commitlint is straightforward, simple and easy. The long-term benefits -- clear commit messages, consistent history -- are well worth the effort.

---

## Branches: Organized and Traceable

In highly critical systems, branch organization is non-negotiable. It's not just about avoiding confusion -- it's about making sure work can be traced back to its purpose, especially in microservices architectures where each service lives in its own repository.

### 1. Follow a Consistent Naming Convention

A good branch name starts with the task or issue identifier, making it easy to see what the branch is for. I always use this format:

**Format:**

```text
<JIRA-ticket-ID>-<type>-<short-description>
```

**Examples:**

- `JIRA-5678-fix-transaction-timeout`
- `JIRA-1234-feature-add-batch-processing`

This convention has saved me -- and my team -- countless hours when tracking work across multiple repositories.

---

### 2. Keep Branches Short-Lived

The longer a branch stays open, the more likely it is to diverge from the base branch. I aim to merge branches into `main` or `develop` frequently, keeping integration smooth and reducing conflicts.

---

### 3. Rebase for a Clean History

Rebasing instead of merging keeps your branch history linear, which is much easier to follow during debugging or reviews.

**Example Workflow:**

```bash
git checkout JIRA-5678-fix-transaction-timeout
git pull --rebase origin main
```

Rebasing has saved me from so many messy histories, but I'm always careful not to rebase shared branches like `main`.

Rebasing rewrites commit hashes, so it should only be used on local or feature branches. The payoff is a linear history that reads like a narrative rather than a tangled graph. During incident investigations, a linear `git log` lets you bisect and isolate the offending change in seconds rather than minutes.

gitGraph
    commit id: "init"
    commit id: "v1.0"
    branch JIRA-5678-fix-timeout
    commit id: "add retry logic"
    commit id: "handle edge case"
    commit id: "add tests"
    checkout main
    commit id: "hotfix: logging"
    checkout JIRA-5678-fix-timeout
    merge main id: "rebase onto main" type: HIGHLIGHT
    checkout main
    merge JIRA-5678-fix-timeout id: "squash merge" type: HIGHLIGHT
    commit id: "v1.1"

---

## Pull Requests: Your Code Documentation

Pull requests are where collaboration happens. In highly critical systems, they also serve as an essential checkpoint to catch mistakes before they make it to production.

### 1. Use Clear and Structured PR Titles

A PR title should be concise but informative. I use this format to keep things consistent and easily traceable:

```text
[JIRA-ticket-ID] <Type>: <Short Description>
```

**Examples:**

- `[JIRA-5678] Fix: Handle transaction timeout edge cases`
- `[JIRA-1234] Feature: Add bulk processing for transactions`

---

### 2. Write Descriptive PR Descriptions

A good PR description provides enough context to help reviewers understand what's changing and why. I try to answer three key questions:

1. **References and Documentation**
2. **What changed?**
3. **Why was this change made?**
4. **Does it introduce any risks or side effects?**

```markdown
### What
Added a batch endpoint for processing transaction summaries.

### Why
This improves efficiency for bulk transaction reconciliation.

### Impact
- Adds a new API endpoint.
- No breaking changes.
- Includes unit and integration tests.
```

---

### 3. Keep PRs Small and Focused

Large PRs are overwhelming to review and prone to mistakes. I aim to keep PRs focused on a single feature, bug, or task to make reviews faster and more effective. It is absolutely okay to have couple of PRs for few critical implementations separated by small subtasks/milestones, in my opinion.

**Example: Schema migration as a separate Pull Request.**

---

### 4. Use Checklists and Labels

Checklists ensure that every step is complete before merging:

- Unit tests added
- Integration tests verified
- Documentation updated

Labels is such an under-rated feature, which I regularly use. Labels like `feature`, `fix`, or `hotfix` also help prioritize reviews and are easy to filter out and dig back when required.

graph LR
    A["Branch Created"] --> B["Commits"]
    B --> C["Self-Review"]
    C --> D["PR Opened"]
    D --> E["CI Passes"]
    E --> F["Code Review"]
    F --> G["Approved"]
    G --> H["Squash Merge"]
    H --> I["Branch Deleted"]

    style A fill:#64748b,color:#fff,stroke:none
    style B fill:#64748b,color:#fff,stroke:none
    style C fill:#64748b,color:#fff,stroke:none
    style D fill:#64748b,color:#fff,stroke:none
    style E fill:#10b981,color:#fff,stroke:none
    style F fill:#64748b,color:#fff,stroke:none
    style G fill:#10b981,color:#fff,stroke:none
    style H fill:#10b981,color:#fff,stroke:none
    style I fill:#64748b,color:#fff,stroke:none

---

## Why These Practices Matter

In **highly critical, production-grade FinTech systems**, precision isn't optional. I've seen how poor Git practices can snowball into major issues -- long debugging sessions, delayed releases, or even customer-facing outages. Clean commits, structured branches, and clear PRs aren't just best practices -- they're safeguards for the stability and trustworthiness of the systems we build.

By following these practices:

- Debugging in production becomes faster and more efficient.
- Collaboration is smoother because everything is easy to trace.
- Compliance and audits are simpler, with clear histories and well-documented changes.

If you've got your own favorite tools or Git horror stories, let's swap notes. The best practices we share today could save someone a lot of time (and stress) tomorrow.


---

## Go Constants Beyond the Basics

URL: https://ashwingopalsamy.in/writing/go-constants-beyond-the-basics/
Published: 2024-11-13
Tags: go


When I first got into Go, I thought constants were simple and limited -- just fixed values, nothing fancy. But as I delved deeper, I found they're quite versatile. Yes, they're fixed values, but Go handles them in ways that are both flexible and efficient. Let's see what that means with some practical examples.

## Constants as Type-Free Values (Until They're Used)

In Go, constants are often untyped until you actually use them. They have a default kind but can be assigned to variables of different types, as long as the value fits. This makes them adaptable in a way that's unusual for a statically typed language.

Here's how that looks:

```go
const x = 10

var i int = x
var f float64 = x
var b byte = x
```

graph TD
    A["const x = 5\n(untyped)"] --> B["used as int"]
    A --> C["used as float64"]
    A --> D["used as byte"]
    B --> E["int"]
    C --> F["float64"]
    D --> G["byte"]

    style A fill:#6366f1,color:#fff,stroke:none
    style B fill:#64748b,color:#fff,stroke:none
    style C fill:#64748b,color:#fff,stroke:none
    style D fill:#64748b,color:#fff,stroke:none
    style E fill:#10b981,color:#fff,stroke:none
    style F fill:#10b981,color:#fff,stroke:none
    style G fill:#10b981,color:#fff,stroke:none

A bit analogous to Schrodinger's paradox, `x` can be an `int`, a `float64`, or even a `byte` until you assign it. This temporary flexibility lets `x` work smoothly with different types in your code. No need for casting, which keeps things neat.

You can even mix constants of different types in expressions and Go will figure out the best type for the result:

```go
const a = 1.5
const b = 2

const result = a * b // result is float64
```

Since `a` is a floating-point number, Go promotes the whole expression to `float64`. So you don't have to worry about losing precision -- Go handles it. But be careful: if you try to assign `result` to an `int`, you'll get an error. Go doesn't allow implicit conversions that might lose data.

**Limitations**

This flexibility only goes so far. Once you assign a constant to a variable, that variable's type is set:

```go
const y = 10

var z int = y       // z is an int
var k float64 = y   // y can still be used as float64
```

But if you try this:

```go
const y = 10.5

var m int = y       // Error: constant 10.5 truncated to integer
```

Go will throw an error because it won't automatically convert a floating-point constant to an integer without an explicit cast. So while constants are flexible, they won't change type to fit incompatible variables.

### Understanding Type Defaults

When you use untyped constants without specifying a type, they assume a default type:

- Untyped Integer Constants default to `int`
- Untyped Floating-Point Constants default to `float64`
- Untyped Rune Constants default to `rune` (which is `int32`)
- Untyped Complex Constants default to `complex128`
- Untyped String Constants default to `string`
- Untyped Boolean Constants default to `bool`

## Compile-Time Evaluation and Performance

Go doesn't just evaluate constants at compile time -- it also optimizes constant expressions. That means you can use constants in calculations, and Go will compute the result during compilation:

```go
const a = 100
const b = 5
const c = a * b + 20 // c is computed at compile time
```

So `c` isn't recalculated at runtime; Go has already figured out it's `520` at compile time. This can boost performance, especially in code where speed matters. By using constants, Go handles the calculations once, instead of doing them every time your program runs.

## Constants in Conditional Compilation

Go doesn't have a preprocessor like some other languages, but you can use constants in `if` statements to include or exclude code at compile time.

```go
const debug = false

func main() {
    if debug {
        fmt.Println("Debugging enabled")
    }
    // The above block might be removed by the compiler if debug is false
}
```

When `debug` is `false`, the compiler knows the `if` condition will never be true and might leave out the code inside the block. This can make your final binary smaller.

## Working with Big Numbers

One powerful feature of Go's constants is that they support very large numbers. Untyped numeric constants in Go have "infinite" precision, limited only by memory and the compiler.

```go
const bigNum = 1e1000 // This is a valid constant
```

Even though `bigNum` is way bigger than any built-in numeric type like `float64` or `int`, Go lets you define it as a constant. You can do calculations with these large numbers at compile time:

```go
const (
    a = 1e20
    b = 1e30
    c = a * b // c is 1e50
)
```

## Typed Constants with iota

If you've been using Go, you've probably seen `iota` for creating enumerated constants. It's useful because it automatically assigns incremental values.

You can also use expressions in constant declarations with `iota` to create related constants.

```go
const (
    _ = iota
    KB = 1 << (10 * iota)
    MB
    GB
    TB
)
```

This code defines constants for kilobyte, megabyte, gigabyte, and terabyte using bit shifting. It's calculated at compile time. It's a neat way to generate a series of related constants.

I find `iota` really helpful for this kind of stuff. As Go doesn't have a built-in enum type, you can effectively simulate enums using the `iota` identifier and custom types.

graph LR
    A["iota=0"] --> B["1<<0 = 1\n(Read)"]
    C["iota=1"] --> D["1<<1 = 2\n(Write)"]
    E["iota=2"] --> F["1<<2 = 4\n(Execute)"]
    B --> G["Read | Write = 3"]
    D --> G

    style A fill:#64748b,color:#fff,stroke:none
    style C fill:#64748b,color:#fff,stroke:none
    style E fill:#64748b,color:#fff,stroke:none
    style B fill:#10b981,color:#fff,stroke:none
    style D fill:#f59e0b,color:#fff,stroke:none
    style F fill:#ef4444,color:#fff,stroke:none
    style G fill:#6366f1,color:#fff,stroke:none

## Constants with Bitwise Operations and Shifts

Constants can use bitwise operations and shifts, even resulting in values that are bigger than any built-in type.

```go
const (
    shiftAmount = 100
    shiftedValue = 1 << shiftAmount // shiftedValue is a huge number (1267650600228229401496703205376)
)
```

Here, `shiftedValue` becomes a very large number because of the big shift amount. This value is too big for standard integer types but is valid as a constant until you try to assign it:

```go
var n int = shiftedValue // Error: constant overflows int
```

This shows that constants can represent values you can't store in variables, allowing for compile-time calculations with very large numbers.

## Limitations with Constants

While Go's constants are flexible, there are some things they can't do.

### 1. Constants Cannot Be Referenced by Pointers

Constants don't have a memory address at runtime. So you can't take the address of a constant or use a pointer to it.

```go
const x = 10
var p = &x // Error: cannot take the address of x
```

### 2. Constants with Typed nil Pointers

While `nil` can be assigned to variables of pointer, slice, map, channel, and function types, you cannot create a constant that holds a typed `nil` pointer.

```go
const nilPtr = (*int)(nil) // Error: const initializer (*int)(nil) is not a constant
```

This adds to the immutability and compile-time nature of constants in Go.

### 3. Function Calls in Constant Declarations

Only certain built-in functions can be used in constant expressions, like `len`, `cap`, `real`, `imag`, and `complex`.

```go
const str = "hello"
const length = len(str) // This works
const pow = math.Pow(2, 3) // Error: math.Pow cannot be used in constant expressions
```

### 4. Composite Types and Constants

Constants can't directly represent composite types like slices, maps, or structs. But you can use constants to initialize them.

```go
const mySlice = []int{1, 2, 3} // Error: []int{…} is not constant
```

The code above doesn't work because you can't declare a slice as a constant. However, you can use constants inside a variable slice:

```go
const a = 1
const b = 2
const c = 3

var mySlice = []int{a, b, c} // This is fine
```

Just remember, the slice itself isn't a constant -- you can't declare it as one. The elements inside can be constants, though.

### 5. Explicit Conversion When Needed

If an untyped constant can't be directly assigned due to a type mismatch or possible loss of precision, you need to use an explicit type conversion.

```go
const y = 1.9999
var i int = int(y) // This works, but you lose the decimal part
```

## Wrapping Up

I hope this gives you a better idea about constants. They're not only simple fixed values, but also a flexible feature that can make your code more expressive and efficient.


---

## Go's UTF-8 Identifier Limitation

URL: https://ashwingopalsamy.in/writing/gos-utf-8-identifier-limitation/
Published: 2024-11-12
Tags: go, unicode


I've been exploring Go's UTF-8 support lately, and was curious about how well it handles non-Latin scripts in code. This post covers a detailed overview about the same.

## Go and UTF-8

We know that Go source files are UTF-8 encoded by default. This means you can, in theory, use Unicode characters in your variable names, function names and more.

For example, in the official Go playground [boilerplate code](https://go.dev/play/), you might come across code like this:

```go
package main

import "fmt"

func main() {
    消息 := "Hello, World!"
    fmt.Println(消息)
}
```

Here, `消息` is Chinese for "message." Go handles this without any issues, thanks to its Unicode support. This capability is one reason why Go has gained popularity in countries like China and Japan -- developers can write code using identifiers meaningful in their own languages.

You won't believe it, but there's some popularity in China, for writing code in their native language and I love it.

flowchart TD
    A["Character"] --> B{"Letter\n(Lu/Ll/Lo)?"}
    B -- Yes --> C["Valid identifier start"]
    B -- No --> D{"Mark/Digit\n(Mn/Mc/Nd)?"}
    D -- Yes --> E["Valid continuation only"]
    D -- No --> F["Invalid in identifiers"]

    style C fill:#10b981,color:#fff,stroke:none
    style E fill:#f59e0b,color:#fff,stroke:none
    style F fill:#ef4444,color:#fff,stroke:none
    style A fill:#64748b,color:#fff,stroke:none
    style B fill:#6366f1,color:#fff,stroke:none
    style D fill:#6366f1,color:#fff,stroke:none

## Attempting to Use Tamil Identifiers

Naturally, I wanted to try this out with Tamil, my mother tongue. [Tamil](https://en.wikipedia.org/wiki/Tamil_language), one of the world's oldest languages, is spoken by over 85 million people globally and uses a non-Latin script quite distinct from widely-used scripts like Chinese. While coding in Tamil isn't common even in regions where it's spoken, its unique structure made it an intriguing choice for my experiment with Go's Unicode support.

Here's a simple example I wrote:

```go
package main

import "fmt"

func main() {
    எண்ணிக்கை := 42 // "எண்ணிக்கை" means "number"
    fmt.Println("Value:", எண்ணிக்கை)
}
```

At first glance, this seems straightforward that can run without any errors.

But, when I tried to compile the code, I ran into errors:

```
./prog.go:6:11: invalid character U+0BCD '்' in identifier
./prog.go:6:17: invalid character U+0BBF 'ி' in identifier
./prog.go:6:23: invalid character U+0BCD '்' in identifier
./prog.go:6:29: invalid character U+0BC8 'ை' in identifier
./prog.go:7:33: invalid character U+0BCD '்' in identifier
./prog.go:7:39: invalid character U+0BBF 'ி' in identifier
./prog.go:7:45: invalid character U+0BCD '்' in identifier
./prog.go:7:51: invalid character U+0BC8 'ை' in identifier
```

### Understanding the Issue with Tamil Combining Marks

To understand what's going on, it's essential to know a bit about how Tamil script works.

Tamil is an [abugida](https://en.wikipedia.org/wiki/Abugida) -- a writing system where each consonant-vowel sequence is written as a unit. In Unicode, this often involves combining a base consonant character with one or more combining marks that represent vowels or other modifiers.

**For example:**

- The Tamil letter `க` (U+0B95) represents the consonant sound "ka"
- To represent "ki" you'd combine `க` with the vowel sign `ி` (U+0BBF), resulting in `கி`
- The vowel sign `ி` is a **combining mark**, specifically classified as a [Non-Spacing Mark](https://www.compart.com/en/unicode/category/Mn) in Unicode

### Here's where the problem arises

Go's language specification allows Unicode letters in identifiers but excludes combining marks. Specifically, identifiers can include characters that are classified as "Letter" (categories `Lu`, `Ll`, `Lt`, `Lm`, `Lo`, or `Nl`) and digits, but not combining marks (categories `Mn`, `Mc`, `Me`).

### Examples of Combining Marks in Tamil

Let's look at how Tamil characters are formed:

**Standalone Consonant:** `க` (U+0B95) -- Allowed in Go identifiers.

**Consonant + Vowel Sign:** `கா` (U+0B95 U+0BBE) -- Not allowed because `ா` (U+0BBE) is a combining mark (`Mc`).

**Consonant + Vowel Sign:** `கி` (U+0B95 U+0BBF) -- Not allowed because `ி` (U+0BBF) is a combining mark (`Mn`).

**Consonant + Vowel Sign:** `கூ` (U+0B95 U+0BC2) -- Not allowed because `ூ` (U+0BC2) is a combining mark (`Mc`).

In the identifier `எண்ணிக்கை` ("number"), the characters include combining marks:

- `எ` (U+0B8E) -- Letter, allowed
- `ண்` (U+0BA3 U+0BCD) -- Formed by `ண` (U+0BA3) and the virama `்` (U+0BCD), a combining mark (`Mn`)
- `ண` (U+0BA3) -- Letter, allowed
- `ிக்கை` -- Contains combining marks like `ி` (U+0BBF) and `ை` (U+0BC8)

Because these combining marks are not allowed in Go identifiers, the compiler throws errors when it encounters them.

graph LR
    A["Tamil syllable"] --> B["Base consonant\n(Lo: valid)"]
    A --> C["Vowel sign\n(Mc: continuation only)"]
    B --> D["Visually incomplete\nalone"]
    C --> D

    style A fill:#64748b,color:#fff,stroke:none
    style B fill:#10b981,color:#fff,stroke:none
    style C fill:#ef4444,color:#fff,stroke:none
    style D fill:#f59e0b,color:#fff,stroke:none

## Why Chinese Characters Work but Tamil Doesn't

Chinese characters are generally classified under the "Letter, Other" (`Lo`) category in Unicode. They are standalone symbols that don't require combining marks to form complete characters. This is why identifiers like `消息` work perfectly in Go.

**Practical Implications**

The inability to use combining marks in identifiers has significant implications for scripts like Tamil:

- Without combining marks, it's nearly impossible to write meaningful identifiers in Tamil
- Using native scripts can make learning to code more accessible, but these limitations hinder that possibility, particularly for languages that follow abugida-based writing systems

## What's wrong here?

Actually, nothing really!

Go's creators primarily aimed for consistent string handling and alignment with modern web standards through UTF-8 support. They didn't **necessarily intend for "native-language" coding** in identifiers, especially with scripts requiring combining marks.

I wanted to experiment how far we could push Go's non-Latin alphabet support. Although most developers use and prefer English for coding, I thought it would be insightful to explore this aspect of Go's Unicode support.


---

## UUIDCheck

URL: https://ashwingopalsamy.in/projects/uuidcheck/
Published: 2024-09-05
Tech: Go
GitHub: https://github.com/ashwingopalsamy/uuidcheck


A lightweight Go package for validating UUID strings and identifying their version and variant. Supports all UUID versions from RFC 4122 (v1-v5) through RFC 9562 (v6, v7, v8), with zero external dependencies.

## Motivation

UUID validation in Go typically means a regex or pulling in a full UUID generation library just to check a string. UUIDCheck does one thing well: tell you if a UUID is valid, what version it is, and whether the variant bits are correct. Nothing more.

## Design

Pure standard library. The parser validates structure (8-4-4-4-12 hex format), extracts version from the 13th nibble, checks variant bits in the clock sequence, and returns a typed result. Useful as a validation layer at API boundaries or in data pipeline ingestion.


---

## UUIDv8

URL: https://ashwingopalsamy.in/projects/uuidv8/
Published: 2024-06-10
Tech: Go
GitHub: https://github.com/ash3in/uuidv8


A Go library for generating UUIDv8 identifiers per RFC 9562. UUIDv8 is the newest UUID version, designed for applications that need custom timestamp encoding, database-friendly sorting, and domain-specific node bits -- all within the standard 128-bit UUID format.

## Why UUIDv8

UUIDv4 is random. UUIDv7 is time-ordered but opinionated about its timestamp layout. UUIDv8 gives you full control over the custom bits while remaining RFC-compliant. This matters when you need deterministic ordering, embedded metadata, or compatibility with systems that validate UUID structure.

## Design

Zero external dependencies. The library handles timestamp encoding, clock sequence management, and node ID generation while enforcing the RFC 9562 variant and version bits. The API is deliberately minimal -- one function to generate, one to parse.


---

## Pismo Zones

URL: https://ashwingopalsamy.in/projects/pismozones/
Published: 2026-03-20
Tech: JavaScript, Vite
GitHub: https://github.com/ashwingopalsamy/pismozones


Timezone tool built for Pismo's engineering team spread across Sao Paulo, Bangalore, Bristol, Austin, and Singapore. AI-engineered from design to deployment. Used daily in production by Pismo engineers to coordinate across five offices.

## Motivation

Distributed teams waste time on timezone math. Every standup, every cross-office sync, someone asks "what time is it there?" Pismo Zones answers that instantly with a visual that updates in real-time.

## Design

Glassmorphism layers, spring-physics card transitions, and a color-interpolated gradient that shifts with time of day. The interface adapts ambient lighting based on the hour. Luxon handles all timezone math to avoid native Date object inconsistencies. Pure CSS variables, no utility frameworks. Client-side only, zero backend.

## Case Study

The design process, color theory, motion physics, and implementation decisions are documented in an interactive case study:

[View Design Case Study](/projects/pismozones/design-case-study/)


---

## Claude Code Theme

URL: https://ashwingopalsamy.in/projects/claude-code-theme/
Published: 2026-02-10
Tech: TypeScript
GitHub: https://github.com/ashwingopalsamy/claude-code-theme


A VS Code theme system built around warm Claude-inspired palettes. 12 variants covering dark, light, brand, high-contrast, and no-bold editions. Published on the VS Code Marketplace.

## Motivation

Most themes are palette swaps. They pick colors and drop them into a JSON file. The result looks fine on one language and breaks on another because nobody tested the interaction between workbench chrome, semantic tokens, and TextMate scopes across languages.

## Design

A programmatic compilation pipeline generates all 12 variants from shared brand tokens. Each variant compiles to 369 workbench colors, 81 TextMate rules, and 37 semantic token rules. CI runs WCAG contrast validation on every build, with editor text hitting 14.92-16.98:1 contrast ratios. TextMate scope packs cover JS/TS, Go, Python, Java, Rust, HTML/CSS, JSON/YAML, Markdown, and SQL.


---

## Claude Code Launcher

URL: https://ashwingopalsamy.in/projects/claude-code-launcher/
Published: 2026-02-02
Tech: Terminal
GitHub: https://github.com/ash3in/claude-code-launcher


A shell utility that reduces enterprise Claude Code setup to one command. Type `cc` and you're in.

## Motivation

Enterprise users accessing Claude Code through corporate endpoints deal with refreshable JWT tokens, multi-step environment variable setup, and token management across sessions. Tokens end up in shell history, visible in process lists. Every new terminal session means pasting a long JWT again.

## Design

A single shell function handles token storage (file permissions 600, never in history or `ps` output), JWT format validation, expiry tracking with days-remaining display, and endpoint configuration. An interactive installer detects your shell, sets up the alias, and optionally adds a prompt indicator. Daily use is one command: `cc`.


---

## Repo Rate Visualizer

URL: https://ashwingopalsamy.in/projects/repo-rate-visualizer/
Published: 2026-02-10
Tech: JavaScript, React
GitHub: https://github.com/ashwingopalsamy/repo-rate-visualizer


Interactive data visualization of India's Reserve Bank of India repo rate decisions over time. Timeline charts, rate change analysis, monetary policy cycle comparisons, and CSV export.

## Motivation

RBI repo rate data is public but scattered across press releases and PDF tables. No single tool lets you see the full timeline, compare easing and tightening cycles, or share a specific date range with a URL.

## Design

D3.js renders the charts from bundled historical data. Multiple views: timeline, rate change bars, cycle comparison, and event annotations. A filter bar supports date range presets (10Y, 5Y, custom) and a URL state hook encodes the current view and range into the address bar for shareable links. Separate mobile layout components handle smaller viewports.


---

## MKKS Organics

URL: https://ashwingopalsamy.in/projects/mkks-organics/
Published: 2026-03-01
Tech: JavaScript, React, Vite
GitHub: https://github.com/ashwingopalsamy/host-mkks-organics


Storefront for MKKS Organics, a family mango orchard business. Customers browse varieties, select quantities, and send a structured reservation request via WhatsApp.

## Motivation

The business needed an online presence but not a full e-commerce stack. No payment gateway, no user accounts, no database. Orders are seasonal, volume is manageable over WhatsApp, and the overhead of maintaining a backend would outweigh its value.

## Design

Static React site on Vercel. Product data and pricing live in a single content file. The reservation sheet collects quantities by variety and pack size, calculates totals, and constructs a pre-formatted WhatsApp message with the order details and delivery info. Optimized images in WebP with responsive srcSet. Framer Motion for page transitions.


---

## ATS LaTeX Resume

URL: https://ashwingopalsamy.in/projects/ats-latex-resume/
Published: 2024-10-27
Tech: LaTeX
GitHub: https://github.com/ashwingopalsamy/ats-latex-resume


A LaTeX resume template built for machine readability. ATS systems parse the generated PDF correctly, and it still looks clean for human reviewers.

## Motivation

Most LaTeX resume templates optimize for aesthetics. The PDF looks great but ATS software fails to extract the text, misreads special characters, or chokes on custom fonts. Your resume gets filtered out before a human sees it.

## Design

Package choices that target specific ATS failure modes. `pdfgentounicode` and `glyphtounicode` provide Unicode mapping so PDF text extraction works correctly. `lmodern` and `charter` are ATS-safe fonts that don't break parsing. `inputenc` with UTF-8 prevents character encoding failures. Tested against ATS parsers including Jobscan and Resumeworded.


---

## Intelligent Traffic Management System

URL: https://ashwingopalsamy.in/projects/itms/
Published: 2021-03-13
Tech: Python
GitHub: https://github.com/ashwingopalsamy/opnsrc-machinelearning-itms-ug-graduation-project


Final year project. A machine learning system that detects vehicles per lane and adjusts traffic signal timing based on real-time density instead of fixed timers.

## Motivation

Fixed-timer traffic signals ignore actual traffic conditions. A lane with three cars gets the same green time as a lane with thirty. The result is unnecessary congestion on busy lanes and wasted green time on empty ones.

## Design

YOLO model trained on the Indian Driving Dataset for vehicle detection. The pipeline runs non-max suppression, counts vehicles per lane, and feeds the count into a signal timing function that allocates green time proportionally. When conditions fall outside normal parameters, the system falls back to static timing.


---

## Bud.ai

URL: https://ashwingopalsamy.in/projects/budai/
Published: 2022-03-03
Tech: Azure, Teams
GitHub: https://github.com/ashwingopalsamy/opnsrc-microsoft-internship-bud.ai-project


A Microsoft Teams bot that gives students a single place for academic information: schedules, attendance, subjects, faculty contacts. Built as part of the Microsoft Future Ready Talent program during COVID remote learning.

## Motivation

Remote learning during COVID pushed students across multiple platforms for different tasks. Class on Zoom, assignments on one portal, schedules on another, faculty contacts somewhere else. The fragmentation hit junior students hardest.

## Design

Azure Bot Framework handles conversation flow. LUIS provides intent recognition so students can ask naturally instead of navigating menus. QnA Maker handles FAQ-style queries. CosmosDB stores academic data. The bot runs as an Azure App Service and integrates directly into Microsoft Teams, where students already spend their day.


---

last-updated: 2026-04-17