Attempting to use particle.io webhook to send tracking data to traccar

Mitch Mitchell6 years ago

I am attempting to use a particle.io electron with the particle asset tracker to send position information to my traccar server. The traccar server is working fine with the traccar android client, but I cannot get data transmitted via particle webhooks to decode properly.

I have tried sending position information via the following:

  1. Chrome browser with URL pasted into address bar -- success, message received and decoded correctly.
  2. CURL on the local host -- message received and responded to but not decoded
  3. Node Red HTTP Request Node -- message received and responded to but not decoded
  4. Particle.io webook -- message received and responded to but not decoded

I send the following HTTP POST :

http://magnoliamanor-traccar.ddns.net:5055?id={{{location.idn}}}&lat={{{location.lat}}}&lon={{{location.lon}}}&accuracy={{{location.acc}}}&altitude={{{location.alt}}}×tamp={{{location.tim}}}

Particle and Node-red both substitute the parameters producing a valid osmand URL request as far as I can see:

POST /?id=77326&lat=33.064682&lon=-97.003609&accuracy=11.047&altitude=125.691002×tamp=2019-03-09T03:15:40Z HTTP/1.1
Content-Length: 13
host: magnoliamanor-traccar.ddns.net:5055
Connection: close

With logging turned up fully I see the following when the message is not decoded (cases 2-4 above):

2019-03-08 21:27:39  INFO: [9a25e1c9] connected
2019-03-08 21:27:39  INFO: [9a25e1c9: 5055 < 47.185.129.126] HEX: 504f5354202f3f69643d3737333236266c61743d33332e303634363832266c6f6e3d2d39372e3030333630392661636375726163793d31312e30343726616c7469747564653d3132352e3639313030322674696d657374616d703d323031392d30332d30395430333a31353a34305a20485454502f312e310d0a436f6e74656e742d4c656e6774683a2031330d0a686f73743a206d61676e6f6c69616d616e6f722d747261636361722e64646e732e6e65743a353035350d0a436f6e6e656374696f6e3a20636c6f73650d0a0d0a31353532313032303539363231
2019-03-08 21:27:39  INFO: [9a25e1c9: 5055 > 47.185.129.126] HEX: 485454502f312e3120323030204f4b0d0a636f6e74656e742d6c656e6774683a20300d0a0d0a
2019-03-08 21:27:39  INFO: [9a25e1c9] disconnected
2019-03-08 21:27:39 DEBUG: Discarded inbound message org.traccar.model.Position@705adde9 that reached at the tail of the pipeline. Please check your pipeline configuration.
2019-03-08 21:27:39 TRACE: [DEBUG] Released scope instance {0} on thread {1}
2019-03-08 21:27:39 TRACE: [DEBUG] Released scope instance {0} on thread {1}
2019-03-08 21:27:39 TRACE: [DEBUG] Released scope instance {0} on thread {1}
2019-03-08 21:27:40 TRACE: [DEBUG] Released scope instance {0} on thread {1}
2019-03-08 21:27:40 TRACE: [DEBUG] Released scope instance {0} on thread {1}
2019-03-08 21:27:51 DEBUG: HikariPool-1 - Pool stats (total=10, active=0, idle=10, waiting=0)

I am running java version 11.0.1 on a Ubuntu server hosted in the cloud.

openjdk 11.0.1 2018-10-16
OpenJDK Runtime Environment (build 11.0.1+13-Ubuntu-3ubuntu3.18.10.1)
OpenJDK 64-Bit Server VM (build 11.0.1+13-Ubuntu-3ubuntu3.18.10.1, mixed mode, sharing)

Startup logging info here for traccar version information:

2019-03-08 21:39:15  INFO: Logging initialized @9308ms to org.eclipse.jetty.util.log.Slf4jLog
2019-03-08 21:39:17  INFO: Operating system name: Linux version: 4.18.0-15-generic architecture: amd64
2019-03-08 21:39:17  INFO: Java runtime name: OpenJDK 64-Bit Server VM vendor: Oracle Corporation version: 11.0.1+13-Ubuntu-3ubuntu3.18.10.1
2019-03-08 21:39:17  INFO: Memory limit heap: 121mb non-heap: 0mb
2019-03-08 21:39:17  INFO: Character encoding: UTF-8 charset: UTF-8
2019-03-08 21:39:17  INFO: Version: 4.3-SNAPSHOT
2019-03-08 21:39:17  INFO: Starting server...
2019-03-08 21:39:17  INFO: jetty-9.4.14.v20181114; built: 2018-11-14T21:20:31.478Z; git: c4550056e785fb5665914545889f21dc136ad9e6; jvm 11.0.1+13-Ubuntu-3ubuntu3.18.10.1
2019-03-08 21:39:18  INFO: Started o.t.w.@427308f8{/,null,AVAILABLE}
2019-03-08 21:39:18  INFO: DefaultSessionIdManager workerName=node0
2019-03-08 21:39:18  INFO: No SessionScavenger set, using defaults
2019-03-08 21:39:18  INFO: node0 Scavenging every 600000ms
2019-03-08 21:39:20  INFO: Started o.e.j.s.ServletContextHandler@4fafd27e{/,null,AVAILABLE}
2019-03-08 21:39:20  INFO: Started ServerConnector@611ff67c{HTTP/1.1,[http/1.1]}{0.0.0.0:8082}
2019-03-08 21:39:20  INFO: Started @14406ms

Any thoughts on what I can do to make this work -- I'm guessing I need to somehow force the connection to stay open longer? I'm a bit surprised by that but I guess its because the osmand protocal is not really a web server perhaps? (totally guessing about this).

Thanks!

Mitch

Mitch Mitchell6 years ago

Well I was able to work around the issue by using a raw tcp connection where I could control the closing of the connection manually -- holding the connection open for a little as 150ms after receiving the response allows the data to be decoded. Unfortunately this means that particle.io webooks cannot be used because they close the connection immediately after receiving the response.

I think this is anomalous behavior since the server has already sent the HTTP/1.1 200 OK message indicating success on the operation. If it isn't able to handle the message it should not send a success response I think.

Ernesto Vallejo6 years ago

I'm not an expert in this situation, but maybe two things I noticed could guide you to solve the issue.

This is what I get using traccar client

POST /?id=663423×tamp=1552093991&lat=25.6701811&lon=-100.2449368&speed=0.0&bearing=0.0&altitude=0.0&accuracy=98.4000015258789&batt=48.0 HTTP/1.1
Content-Type: application/x-www-form-urlencoded
User-Agent: Dalvik/2.1.0 (Linux; U; Android 8.1.0; Moto G (5) Build/OPPS28.85-13-4)
Host: sub.domain.tld:5055
Connection: Keep-Alive
Accept-Encoding: gzip
Content-Length: 0

And this is what your message states is receiving

POST /?id=77326&lat=33.064682&lon=-97.003609&accuracy=11.047&altitude=125.691002×tamp=2019-03-09T03:15:40Z HTTP/1.1
Content-Length: 13
host: magnoliamanor-traccar.ddns.net:5055
Connection: close

1552102059621

I noted two things:

1- Your timestamp in the string is in ISO Format, when in my example is unix format.
2- I see a timestamp after Connection header and a line break.

Could this be also a problem?
are you missing any headers?

Hope this helps.

Mitch Mitchell6 years ago

Yes the ISO format string is now supported in Traccar, but I tried both formats just to be sure -- no effect.
The timestamp in the HTTP body is just an artifact of how the Node-Red flow triggered the HTTP request, the msg.payload field in Node-Red held a timestamp from the Inject node so the Http Request node places it in the request body. I took it out and tested too and it made no difference.

What finally fixed it was holding the connection open for about 150ms after the HTTP 200 reply was received. I think that is a bug, but I've seen that behavior in a lot of IoT systems -- the standard allows the HTTP client to close the connection immediately after sending its payload, it doesn't even have to wait for a reply but many systems seem to not handle that correctly.

Mitch Mitchell6 years ago

It looks like this is behavior in netty -- I don't know whether the traccar team can do anything other than perhaps not send the HTTP 200 message until the incoming payload is processed.
https://github.com/netty/netty/issues/7285