Hi everyone,
We ran into an issue recently, and I decided to share it here in case it helps others.
Description
Sending an email notification blocks the thread that dispatches notifications
for an event, so a slow or unreachable SMTP server delays (or effectively
blocks) delivery of other notification channels for the same event —
including web and firebase, which have nothing to do with email.
I reproduced this on v6.9.1, where NotificatorMail.send() is fully
synchronous (throws MessageException, no async wrapper at all):
https://github.com/traccar/traccar/blob/v6.9.1/src/main/java/org/traccar/notificators/NotificatorMail.java#L33
The abstract Notificator.send(Notification, User, Event, Position) calls
this directly, with no executor/queue in between:
https://github.com/traccar/traccar/blob/v6.9.1/src/main/java/org/traccar/notificators/Notificator.java#L28-L31
I checked current master (as of writing, ~v6.14.5) and the pattern is
still effectively the same, just with a different signature. sendAsync()
now returns a CompletableFuture<Void>, but the SMTP send still happens
inline before the future is constructed — there's no
CompletableFuture.supplyAsync() / executor involved, so the "async" is in
name only:
https://github.com/traccar/traccar/blob/master/src/main/java/org/traccar/notificators/NotificatorMail.java#L44
Impact
When the SMTP server is unreachable (wrong host, firewall blocking egress,
provider outage, etc.), Transport.connect() blocks on the calling thread
until the underlying TCP connect attempt times out — which can take
several minutes depending on the network stack, since JavaMail never
receives an explicit timeout (see related issue below). Every notification
dispatch that includes mail as one of its channels pays this cost, even
for channels like web/firebase that would otherwise be instant.
In our case this caused real-time web notifications to arrive anywhere
from a few seconds to 10+ minutes late, all traced back to intermittent
SMTP timeouts on one mail provider.
Related issue
mail.smtp.timeout and mail.smtp.connectiontimeout are not included in
the property whitelist in SmtpMailManager.getProperties():
https://github.com/traccar/traccar/blob/master/src/main/java/org/traccar/mail/SmtpMailManager.java#L60-L74
So even setting these keys in traccar.xml has no effect — JavaMail falls
back to its own (very long, or effectively unbounded) default timeouts.
Suggested fix
- Run the mail send on a dedicated executor so a stuck SMTP call can't
block dispatch of unrelated notification channels.
- Add
MAIL_SMTP_TIMEOUT / MAIL_SMTP_CONNECTIONTIMEOUT to the
whitelist in SmtpMailManager.getProperties(), with sane defaults
(e.g. 5000ms), so traccar.xml config actually takes effect.
Steps to reproduce
- Configure
mail.smtp.host pointing to an unreachable host/port (e.g.
blocked by firewall so the TCP SYN just hangs).
- Trigger any event with a notification enabled for both
mail and
web/firebase.
- Observe the
web/firebase notification is delayed until the SMTP
connection attempt times out, visible in logs as:
Connection timed out - ConnectException (... < SmtpMailManager:154 ... < NotificatorMail:44/47 < Notificator ...)
Environment
- Reproduced on: v6.9.1
- Confirmed same underlying issue present on: master (v6.14.5)
- OS: Ubuntu 24.04
Hi everyone,
We ran into an issue recently, and I decided to share it here in case it helps others.
Description
Sending an email notification blocks the thread that dispatches notifications
for an event, so a slow or unreachable SMTP server delays (or effectively
blocks) delivery of other notification channels for the same event —
including
webandfirebase, which have nothing to do with email.I reproduced this on v6.9.1, where
NotificatorMail.send()is fullysynchronous (throws
MessageException, no async wrapper at all):https://github.com/traccar/traccar/blob/v6.9.1/src/main/java/org/traccar/notificators/NotificatorMail.java#L33
The abstract
Notificator.send(Notification, User, Event, Position)callsthis directly, with no executor/queue in between:
https://github.com/traccar/traccar/blob/v6.9.1/src/main/java/org/traccar/notificators/Notificator.java#L28-L31
I checked current
master(as of writing, ~v6.14.5) and the pattern isstill effectively the same, just with a different signature.
sendAsync()now returns a
CompletableFuture<Void>, but the SMTP send still happensinline before the future is constructed — there's no
CompletableFuture.supplyAsync()/ executor involved, so the "async" is inname only:
https://github.com/traccar/traccar/blob/master/src/main/java/org/traccar/notificators/NotificatorMail.java#L44
Impact
When the SMTP server is unreachable (wrong host, firewall blocking egress,
provider outage, etc.),
Transport.connect()blocks on the calling threaduntil the underlying TCP connect attempt times out — which can take
several minutes depending on the network stack, since JavaMail never
receives an explicit timeout (see related issue below). Every notification
dispatch that includes
mailas one of its channels pays this cost, evenfor channels like
web/firebasethat would otherwise be instant.In our case this caused real-time web notifications to arrive anywhere
from a few seconds to 10+ minutes late, all traced back to intermittent
SMTP timeouts on one mail provider.
Related issue
mail.smtp.timeoutandmail.smtp.connectiontimeoutare not included inthe property whitelist in
SmtpMailManager.getProperties():https://github.com/traccar/traccar/blob/master/src/main/java/org/traccar/mail/SmtpMailManager.java#L60-L74
So even setting these keys in
traccar.xmlhas no effect — JavaMail fallsback to its own (very long, or effectively unbounded) default timeouts.
Suggested fix
block dispatch of unrelated notification channels.
MAIL_SMTP_TIMEOUT/MAIL_SMTP_CONNECTIONTIMEOUTto thewhitelist in
SmtpMailManager.getProperties(), with sane defaults(e.g. 5000ms), so
traccar.xmlconfig actually takes effect.Steps to reproduce
mail.smtp.hostpointing to an unreachable host/port (e.g.blocked by firewall so the TCP SYN just hangs).
mailandweb/firebase.web/firebasenotification is delayed until the SMTPconnection attempt times out, visible in logs as:
Connection timed out - ConnectException (... < SmtpMailManager:154 ... < NotificatorMail:44/47 < Notificator ...)Environment