6.14.5 — positions silently stop being stored for most devices (devices stay online), immediate on upgrade from 5.10; reverting to 5.10 fixes instantly

Upgraded a production server from 5.10 to 6.14.5 (installer, Liquibase migration clean, PostgreSQL). From the very first hour, position storage collapsed while devices stayed connected:

• Fleet: ~540 registered devices, ~230 active (gt06, startek, osmand). PostgreSQL 8GB droplet.
• Healthy on 5.10: ~45–50k positions/hour, ~165 distinct devices/hour.
• On 6.14.5: hourly rate decayed 22k → 15k → 10k → 5k over 10 hours; distinct devices with a fresh fix fell to ~19 and stayed there. The surviving ~19 were a fixed subset (mixed protocols, mixed user counts — no pattern we could find).
• Devices remained CONNECTED the whole time: 165 devices with lastupdate < 10 min, heartbeats flowing, /api/devices reporting them online — but their positions were not stored. No decoder errors, zero "filtered" log lines (filter config unchanged from 5.10), geocoder healthy (0.12s responses).
• gt06 rows that did get stored had a median servertime - fixtime lag of ~16 hours — i.e., only buffered replays were being accepted; live gt06 traffic effectively dead. startek rows that got stored had lag 0 but the set of startek devices storing anything shrank hour by hour.
• systemctl restart traccar did NOT recover it — connections re-established (148), but the same ~19-device subset kept storing and everyone else stayed frozen.
• Thread dump (kill -3) during the freeze: 52 threads, none BLOCKED, nothing stuck in geocoding or storage — looks like a logic path silently skipping the store, not a deadlock.
• Reverting the binary to 5.10 on the SAME migrated database recovered ingestion instantly (17 → 110+ fresh devices in minutes).

Separate defect found on 6.14.5 with the migrated schema: every audit insert fails —
INSERT INTO tc_actions(actionType, actionTime, address, attributes, userId, objectId, objectType) ... ERROR: integer out of range (PostgreSQL), logged as "Failed to store action create".

This resembles the old "device freezing" threads (6.3/6.4 era) but with restart NOT clearing it. Happy to provide the full thread dump, per-protocol hourly counts, and config (filter.enable=true, filter.invalid=true, geocoder nominatim with ignorePositions=false). What logging should we enable to catch the drop point if we attempt 6.x again?

Possible root cause found. Our config geocodes every position (geocoder.ignorePositions false) against a self-hosted Nominatim. If 6.14.5 processes positions in per-device queues that advance only on the geocoder callback, and the shared HTTP client has no timeouts (added in commit 17292f98 on 30 July, after 6.14.5), then a single hung geocoder request would permanently pin that device's queue — silently, matching everything we saw: devices die one by one, stay online via heartbeats, no errors, no blocked threads, restart re-freezes within minutes. Can you confirm whether that commit addresses this, and whether the fix will be in the next release? Would geocoder.ignorePositions=true be a safe interim workaround on 6.14.5?

It should not freeze permanently. I would definitely recommend disabling automatic geocoding for all positions.

you were right, it wasn't the geocoder. We reproduced the issue on a clean install and found the actual cause:

ROOT CAUSE FOUND — this was a behavior change in FilterHandler between 5.x and 6.14.5, triggered by our own legacy config.

Our traccar.xml carried <entry key='filter.maxSpeed'>0</entry> from 5.10, where 0 meant disabled (5.10: if (filterMaxSpeed != 0 && last != null)). In 6.14.5 the check is if (filterMaxSpeed != null && last != null) with the value coming from AttributeUtil.lookup(...) — so a configured 0 becomes an active 0-knot limit: every position from a vehicle that moved between consecutive fixes is filtered, stationary positions pass. That silently discarded all moving traffic fleet-wide from the first minute after our upgrade, while devices stayed online via heartbeats. It also explains our ~19 "survivors" they were all parked.

Minimal reproduction on a clean 6.14.5 install: one osmand device, send a position with speed > 0 → INFO: Position filtered by MaxSpeed filters from device: X, nothing stored. Remove the filter.maxSpeed entry → everything stores correctly (verified with parked, moving, and fast positions).

Suggestion: restore zero-disables semantics or reject 0 at config validation anyone upgrading from 5.x with this line gets silent fleet-wide position loss.

Fixed:

https://github.com/traccar/traccar/commit/13b1dfb719186ca20f22c01fa3894af9347f2e0c

Closing the loop for anyone who finds this thread later:

We re-ran the upgrade after removing the filter.maxSpeed line from traccar.xml entirely, and it went cleanly this time. Moving vehicles started storing positions within minutes of startup, and after 24+ hours on 6.14.5 the fleet is fully healthy: position rates match our 5.10 baseline (~50k/hour, ~165 active devices), all three protocols we run (gt06, startek, osmand) are live, and zero positions filtered by MaxSpeed.

Summary for others upgrading from old versions: if your config carries filter.maxSpeed set to 0 (which meant "disabled" in older versions), delete the line before upgrading. Anton's fix (commit 13b1dfb) restores the zero-disables behavior for all filters in the next release, so this only affects current builds.

Thanks Anton for the quick fix.

Perfect, great diagnosis @Mia113, thanks for sharing.