DEV Community: Anatolii

Why I set `dynamic: false` on my Elasticsearch audit-log stream

Anatolii — Wed, 24 Jun 2026 12:31:00 +0000

Most Elasticsearch advice is about getting more out of it: better relevance, faster queries, smarter aggregations. This is about the opposite. It's a place where I deliberately told Elasticsearch to do less, and why that was the right call.

👌 The short version: I built an audit-log pipeline, a record of what changed on our entities over time, and I set dynamic: false on the mapping so Elasticsearch would stop trying to index every field it had never seen before. Run an append-only log of arbitrary change events through Elasticsearch with the default settings and the mapping grows without bound. A growing mapping doesn't announce itself. You find out it's a problem once it's already hurting you.

What we were building

The job was an audit log: track what changed on records over time. An entity gets created, updated, deleted, and we write an event describing that change. Which entity, what fields, the old and new values, who did it and when. It's append-only by nature. You never update an audit event, you only add more of them. And you read it far less often than you write it, mostly when someone asks "what happened to this record, and when?"

That shape matters, because it's the shape that makes Elasticsearch's friendliest default turn against you.

The default that will quietly hurt you

By default, Elasticsearch uses dynamic mapping. A document arrives with a field Elasticsearch hasn't seen, it guesses a type, and it adds that field to the index mapping automatically. For most use cases this is genuinely great. You throw documents at it, it figures out the schema, you move on.

Now think about what that means for an audit log.

An audit log's payload is, by design, the union of every field that has ever changed on every entity you track. Every entity type, every column, every nested attribute. Anything that can change is something that can show up in a change event. Run that through dynamic mapping and Elasticsearch faithfully adds a field to the mapping for each new leaf it sees. The mapping stops describing one document shape and slowly accretes every shape your application has ever produced.

A normal index has a roughly stable set of fields. An audit log over a real application does not. It trends toward "every field in the system, ever." So the feature that's a convenience everywhere else is, for this one data shape, a slow leak that you only opt out of if you saw it coming.

What a mapping explosion actually costs

"Mapping explosion" sounds dramatic, so I want to be concrete about the mechanism. The cost is real and it isn't obvious.

The mapping is part of the cluster state, the metadata that lives in memory and is replicated across the cluster. It isn't free per field. As the field count climbs into the thousands, that metadata grows, cluster-state updates get heavier, and the cost is paid by the whole cluster, not just the one index. That's why Elasticsearch ships with a default limit of 1,000 fields per index (index.mapping.total_fields.limit). The limit isn't arbitrary. It's a guardrail against exactly this.

When you hit that limit, you don't get a gentle warning. Indexing fails and the document is rejected. For an audit log that's a nasty failure mode, because the whole point of the system is that it captures everything, and the way dynamic mapping breaks is by silently raising the field count until writes start getting rejected. You can "fix" it by raising the limit, but that's treating the symptom. You're buying time until a bigger number while making every cluster-state update heavier along the way.

There's a second, subtler cost before you ever hit the limit. Every dynamically mapped field is a field Elasticsearch builds indexing structures for. You're paying, in memory and in indexing work, to make fields queryable that, for an audit log, almost nobody will ever query directly.

The decision: `dynamic: false` 🥳

So I locked the mapping down. The setting that matters:

{
  "mappings": {
    "dynamic": false,
    "properties": {
      "@timestamp":  { "type": "date" },
      "entity_id":   { "type": "keyword" },
      "entity_type": { "type": "keyword" },
      "trace_id":    { "type": "keyword" },
      "author": {
        "dynamic": false,
        "properties": {
          "id":       { "type": "keyword" },
          "username": { "type": "keyword" },
          "source":   { "type": "keyword" },
          "name":     { "type": "text" }
        }
      }
    }
  }
}

Notice the same dynamic: false on the nested author object. The author of a change is a small, known set of fields, id, username, source, name, so I map those and lock that sub-object down too. Nothing unexpected creeps in there either.

It's worth being precise about what dynamic: false does, because there are three options and they aren't the same:

dynamic: true (the default): unknown fields get auto-mapped and become searchable. This is the explosion path.
dynamic: false: unknown fields are stored in _source and returned with the document, but they aren't added to the mapping and aren't indexed, so you can't search or aggregate on them. The document is accepted. The field just isn't queryable.
dynamic: strict: a document containing an unknown field is rejected outright.

I chose false, not strict, on purpose. strict would have meant that every time the application started tracking changes on a new field, the audit pipeline would start rejecting events until someone updated the mapping. That couples the audit log's health to every schema change upstream. It's fragile, and it defeats the point of an audit log, which is to never drop a change event. false keeps the full payload in _source, so the complete record of what changed is still there and still retrievable. It just isn't turned into a searchable field. I explicitly map the handful of fields I actually query on, the entity identity, the timestamp, a trace id that ties an event back to the request that caused it, and who made the change, and everything else rides along in _source without bloating the mapping.

That's the whole trade in one sentence: I keep the data, I drop the ability to query arbitrary fields, and in exchange the mapping stays small and the cluster stays healthy.

The trade I accepted

I want to be honest about the cost, because dynamic: false isn't free.

What I gave up: I can't run an ad-hoc query like "find every audit event where some arbitrary nested field changed to value X," because that field was never indexed. If I genuinely needed that, I'd have to map the field explicitly, ahead of time.

Why it was the right trade for this data: an audit log is overwhelmingly read by entity. The real query is "show me the change history for this record," and that's served by the fields I did map, the entity type and id and timestamp. The rare case where someone wants the full detail of a specific event is served by _source, which still has everything. I'm not losing information. I'm losing a query pattern I don't actually use, in exchange for a system that doesn't degrade as the application grows.

The general principle I took from it: dynamic mapping is a convenience that assumes a bounded set of fields. The moment your data's field set is unbounded by design, and an audit log is the textbook example, that convenience works against you, and you should opt out deliberately rather than discover the field limit in production.

Where this lives: data streams, not a plain index

One more piece, because "set dynamic: false" is only half the design. An audit log is append-only and time-ordered. You write a continuous stream of events and basically never modify what's already there. That's the case Elasticsearch data streams are built for.

Instead of writing to a single ever-growing index, a data stream sits in front of a series of backing indices and rolls over to a new one as they grow, while you read and write through one stable name. You configure the whole thing, the dynamic: false mapping included, once, in an index template, and every backing index the stream creates inherits it. So the mapping discipline isn't something you have to remember to reapply. It's in the template, and every new index the stream rolls into starts out locked down.

That pairing is the actual design: a data stream for append-only, time-series writes, plus an index template that carries the dynamic: false mapping so the field set stays controlled no matter how long the log runs or how many indices it rolls through.

The takeaway 📝

If you're putting an append-only log into Elasticsearch, audit events, entity-change history, anything where the field set grows with your application, decide your mapping strategy before the field count forces the decision for you. The default will happily index every field your app has ever produced, and the way you find out is the day writes start failing.

dynamic: false plus a small set of explicitly mapped query fields, sitting on a data stream driven by an index template, is the configuration I'd reach for again without thinking twice. It's a small, boring decision, and boring is what you want here. It's the difference between an audit log that quietly runs for years and one that turns into a cluster-state problem you have to firefight later.

From PHP to Go: what took me longest to rewire

Anatolii — Mon, 22 Jun 2026 20:16:03 +0000

I wrote PHP for about seven years before Go became my main language — Laravel for five of them, Yii2 and plain MVC before that. Then I led the rebuild of a Laravel monolith into Go microservices, and later joined a marketplace-product, to work on Go services in production. So I didn't come to Go from a tutorial. I came to it carrying a decade of PHP habits, and I had to ship real systems while unlearning them 🥲

The syntax was the easy part. You can read Go in an afternoon. What took months to rewire were the mental models — the default assumptions PHP had built into me about how a program is shaped, how errors move, how a request lives and dies. Some of those assumptions are actively wrong in Go, and they don't announce themselves 😫. They show up as code that compiles, passes review on a tired day, and then behaves in a way you didn't predict 😳

This is a list of the ones that took me longest. Each is tied to something I actually built, not a textbook example.

1. There is no `try/catch`, and that is a feature, not a missing one

In PHP I threw exceptions and caught them somewhere up the stack — often far up the stack, in a global handler that turned anything unexpected into a 500. The mental model is: errors travel invisibly until someone decides to look. Most of my code didn't think about failure at all; failure was something that happened to the call stack, above me.

Go inverts this. A function that can fail returns an error as its last value, and you, the caller, deal with it right there 🥳:

user, err := repo.FindUser(ctx, id)
if err != nil {
    return fmt.Errorf("find user %d: %w", id, err)
}

My first instinct was that this was noise. Coming from try { ... } catch (\Throwable $e) {}, the if err != nil after every call felt like ceremony 😅. It took me a while — and a few production incidents — to understand what it buys you: failure becomes part of the visible control flow. You can't not see that a call can fail, because the error is sitting in a variable in front of you. The decision "swallow this, retry this, or pass it up" is made at the exact place that has the most context to make it.

The habit that took longest to kill was the urge to build a catch-all. In PHP I leaned on the global exception handler. In Go I had to learn to wrap errors with context as they go up (%w and a short message at each layer) so that by the time an error reaches the top, the message is a breadcrumb trail — "find user 42: query timeout: ..." — instead of a stack trace I have to decode 🥳

This paid off in a real incident. An order wouldn't create, and the wrapped error that came out the top read essentially like this:

create order 8842: charge payment: gateway timeout after 5s

That one line pointed straight at the cause — a downstream service we were calling to charge the payment was misbehaving and timing out. No log spelunking, no guessing which layer failed. The message had assembled itself from each layer adding a little context on the way up, and by the time it reached me it told me exactly where to look. In my PHP days that would have surfaced as a generic 500 and a stack trace I'd have to read backwards 🥲

What I'd tell a PHP developer: stop looking for try/catch. The if err != nil is your error handling, and writing it everywhere forces you to actually think about each failure instead of deferring all of them to one handler that prints "Something went wrong." 👍

2. The request is not the unit of life anymore

This was the deepest shift, and the one I underestimated most.

In PHP, the model is shared-nothing per request. A request comes in, the framework boots, you handle it, the process tears everything down, and the next request starts from a clean slate. Memory leaks barely matter — the worst case is one request. Global state is reset for you. You almost never think about two requests touching the same variable, because in the classic PHP model they physically can't; they're separate processes 🙂

Go is the opposite. The process is long-lived. One Go binary stays up and handles thousands of requests concurrently, in the same memory, often literally at the same time across goroutines. The moment I internalized that, a whole category of bugs I'd never had to think about became something I had to actively watch for:

A package-level variable is now shared across every concurrent request. In PHP that was a per-request convenience. In Go it's a data race waiting to happen.
A map written to by two requests at once will crash the whole process — not the one request, the whole binary. The PHP blast radius of "one bad request" doesn't exist; a panic from a concurrent map write can take down everything in flight 😢
Resources you open have to be closed deliberately, because nothing is tearing the world down after each request to clean up after you.

None of those are exotic. They're the baseline things you now have to keep in your head on every change, because the language and the runtime won't reset the world for you between requests the way PHP did. The point isn't a specific disaster — it's that a whole class of failure that was simply impossible in the per-request PHP model is now possible by default, and avoiding it is on you 🧐

Rewiring this meant changing my default question. In PHP I asked "what does this request need?" In Go I had to ask "what happens when a thousand of these run at once, in the same memory?" That question is now automatic, but it took real production exposure to make it automatic.

3. Concurrency is in the language, so you own correctness

PHP's concurrency story, for most of my career, was "use a queue and more workers." Parallelism lived outside the language — in the infrastructure, in separate processes, in something like a job queue. I rarely reasoned about two things touching the same data in the same memory, for the same reason as above: they usually couldn't.

Go puts concurrency in my hands directly. go someFunc() starts a goroutine 👍. Channels pass data between them 👍. It's genuinely powerful, and it's the reason Go fit the kind of services we were building 🥳. But the power comes with ownership: the language hands you concurrency and then holds you responsible for correctness. A goroutine that writes to a shared structure without coordination is a bug that may pass every test on your machine and only surface under real load 🥲.

Two specific habits I had to build that PHP never required:

Reach for the race detector early. In my own projects I run tests with -race, and it's caught real races a few times now — races I'd never have spotted by reading the code, because they only show up when goroutines happen to interleave the wrong way at run time 🙏. In PHP I never had a tool like that, because I never had the problem.
Decide deliberately how goroutines share data — pass copies, use a channel, or protect shared state with a mutex — instead of just sharing a variable because it's in scope. "It's in scope so I'll use it" is a perfectly safe PHP habit and a dangerous Go one ☝️

The mental shift: in PHP, concurrency was an infrastructure concern I delegated. In Go, it's a code concern I own line by line.

4. `context.Context` is the spine, not a parameter you tolerate

When I first saw ctx context.Context as the first argument of seemingly every function, I treated it the way I'd treated similar things in PHP frameworks — boilerplate to thread through and otherwise ignore. That was wrong, and it cost me before it clicked 😅.

In PHP, the request was bounded for me. When the client disconnected or the request finished, the process ended; I never had to manually propagate "this work should stop now." In a long-lived Go service, nothing stops your work automatically. If a client gives up, or a request times out, the goroutines doing the work for that request will happily keep running — querying the database, calling other services — for no one. context.Context is how cancellation and deadlines travel down through every call so that work can actually be stopped and resources released.

Once I understood it as the cancellation and deadline spine of the request, passing ctx everywhere stopped feeling like boilerplate and started feeling like the thing that keeps a long-lived service from leaking work 🙌. In a marketplace with a lot of concurrent traffic, "stop doing work nobody is waiting for" is not a nicety; it's how the service stays healthy under load.

A concrete way I use it: when I call another service, I put a deadline on the context — lets say around 5 seconds. If that downstream call runs past the deadline, the context fires, I stop waiting, and I return a clean degraded response to the client — something like "we can't process this right now, please try again in a few minutes" 🤔 — instead of leaving the request hanging forever and tying up resources behind it. That's the whole point of the spine: the deadline travels down with the call, and when it expires everyone downstream can give up together. Under load, failing fast and politely is far better than hanging, and context.Context is what makes that possible without threading a timeout flag through every function by hand 👌

5. Composition over inheritance — and Laravel had hidden how much I leaned on inheritance

This one was subtle because I didn't realize how much of my PHP design instinct was inheritance-shaped until Go took the option away 😎

In Laravel, so much is built on extending base classes — your controllers, your models, your form requests all inherit a large amount of behavior from the framework. The "right" way to add capability was often "extend the base class." That instinct is invisible while you have it; it just feels like how code is organized.

Go has no class inheritance (in usual meaning). It has struct embedding and, more importantly, interfaces that are satisfied implicitly — a type implements an interface just by having the right methods, with no implements keyword and no declared relationship. Coming from PHP's explicit class Foo extends Bar implements Baz, implicit interfaces felt almost too loose at first. Where's the contract? Who guarantees it? 🤯

What rewired it for me was using this pattern consistently at the marketplace-project. There, I define interfaces at the consumer for repositories, for internal services, and for external services — a small interface declared where it's used, and any type with the right methods satisfies it. The consumer declares only the handful of methods it actually needs; the provider doesn't have to know the interface exists. That's a very different shape from "everyone extends the framework's base class," and it produces code where dependencies are small and swapping an implementation is trivial — a real database behind a repository in production, a fake satisfying the same interface in a test. Writing tests stopped requiring a whole framework's worth of scaffolding and started being a matter of passing in a small fake that fits the interface 🥳

Coming to that from years of Laravel's extends-everything instinct is exactly why I can feel how different it is. A developer who only ever wrote Go might take implicit interfaces for granted; I had to consciously give up the inheritance reflex to get there.

6. Explicit beats clever, and the language enforces it

PHP let me be clever. Dynamic typing, magic methods, arrays that were lists and maps and objects depending on my mood — a lot of expressiveness, and a lot of rope. I wrote some clever PHP I was proud of and later could not fully reconstruct why it worked 😅

Go is deliberately boring in a way that annoyed me at first and that I now value. No magic methods. No implicit type juggling. The compiler refuses unused variables and unused imports. The formatting is not up for debate — gofmt decides, and the entire community's code looks the same. Coming from PHP's freedom, this felt like the language not trusting me 🤔

The reframe: Go optimizes for the code being read, not the code being written. On a team — and I was leading one through the rebuild, plus interviewing developers now — that tradeoff is obviously correct. Clever PHP is a liability the moment someone other than its author has to maintain it. Boring, explicit, uniformly-formatted Go is something a teammate can read at 2am during an incident and actually understand. The language took away cleverness, and what I got back - a code that my team could understand about the same way I do 😎

That, more than any single feature, is the mental model I'd most want a PHP developer to adopt before they write a line of Go: you are not writing for the interpreter's flexibility anymore. You are writing for the next person who reads it, and the language is going to hold you to that whether you like it or not 🧐

What I'd actually tell someone making this move

The syntax will take you a week. The mental models took me months, because the hard part isn't learning Go — it's unlearning the PHP assumptions that are so deep you don't know they're assumptions.

None of this is a complaint about PHP. Seven years of PHP is exactly what made the Laravel-to-Go migration something I could lead rather than just attend — I understood the system we were leaving as deeply as the one we were building. But the move only worked once I stopped writing Go in PHP's syntax and started actually thinking in Go.