Replacing backend DB polling with a pub/sub broker

I wanted to fix the other half of the real-time problem this time…

Previously, the browser was already using SSE, but the backend was still polling PostgreSQL every 500ms just to check whether new logs existed or not. It worked, but it felt wasteful. The shape was basically streaming from backend to browser, but still polling from backend to database.

So I replaced that backend-side polling with a small pub/sub broker. The job now publishes log events, the SSE handler subscribes to them, and the broker sits in the middle as the connector between the sender and the receiver.

That is also how I started understanding what a broker really is… basically a middleman. Instead of the producer needing to know who is listening, it can just publish an event, and the broker forwards it to whoever is subscribed. I learned the shape of this from the official NATS pub/sub docs, since NATS is one of the more popular broker implementations and its explanation made the idea click for me.

There were also a few small details here that I liked. Subscribe() and Unsubscribe() use Lock() and Unlock() because they modify the shared map of subscribers, and map is not safe to access concurrently from multiple goroutines. publish() only uses RLock() and RUnlock() because it only reads that map, and with RWMutex multiple readers are allowed at the same time.

What I like about this change is that the database is no longer being asked every 500ms for every open log page just to discover whether nothing changed. The database still matters for persistence and reconnect cases, but live delivery now comes from the broker in memory, which feels much closer to what I actually wanted from the start.

So this step made the architecture cleaner in my head too… PostgreSQL stores the history, while the broker handles the real-time fan-out. That separation feels a lot better than making the database do both jobs.