Author: Richard
[Added to the original trac issue at 3.18pm, Friday, 13th March 2009]

The OSM database doesn't import sub-second timestamps on GPS tracks: anything after the second will be truncated.

Potlatch's GPS display (when pressing 'g' - swf_controller.rb) draws lines between points by timestamp. However, where two points have the same timestamp - as these do after the sub-second portion has been truncated - it won't draw a line. This was added to avoid the case where people upload GPS traces with the same (faked) timestamp on every point, which otherwise makes an almighty mess of the display.

I guess the best solution would be for Potlatch to only draw a dot, not a line, in this case.

Author: eddyp
[Added to the original trac issue at 3.49pm, Friday, 13th March 2009]

Replying to [comment:1 Richard]:

The OSM database doesn't import sub-second timestamps on GPS tracks: anything after the second will be truncated.

That is really a bad decision. The tracks which we have from Norc are sampled at 5 samples/second and they have corrections (dgps) so they are really accurate.

Is there a way to fix this now, or do the tables need to altered? The downloaded trace contains the subseconds, so the data is correctly stored, so I guess all the interfaces need to be fixed (since some are working - see edit trace).

Potlatch's GPS display (when pressing 'g' - swf_controller.rb) draws lines between points by timestamp. However, where two points have the same timestamp - as these do after the sub-second portion has been truncated - it won't draw a line. This was added to avoid the case where people

OK, but why are entire long sections missing (see trace 322818 in the snapshot)?

upload GPS traces with the same (faked) timestamp on every point, which otherwise makes an almighty mess of the display.

I think that someone used an unappropriately sized hammer to fix a issue which doesn't seem hard to fix otherwise (find identically timed points and eliminate/ignore them).

This sounds fixable, can it be done?

I guess the best solution would be for Potlatch to only draw a dot, not a line, in this case.

That would not be a good idea for traces which intersect themselves since the direction of a connection is not clear at all if only dots are drawn.

I think the best solution is to actually allow such high resolutions, not allow points with the same timestamp and use the same trace rendering path for regular edit as is used when 'trace edit'-ing.

Note that I disagree this is a minor bug or even an enhancement since clearly valid and acurate real data is truncated. I will change this, unless some reasonable explanation for the priority downgrade and type downgrade is provided since it helps nobody to hide the dirt under the rug.

Author: Richard
[Added to the original trac issue at 12.11am, Monday, 16th March 2009]

Replying to [comment:2 eddyp]:

Replying to [comment:1 Richard]:

The OSM database doesn't import sub-second timestamps on GPS tracks: anything after the second will be truncated.

That is really a bad decision. The tracks which we have from Norc are sampled at 5 samples/second and they have corrections (dgps) so they are really accurate.

Is there a way to fix this now, or do the tables need to altered? The downloaded trace contains the subseconds, so the data is correctly stored, so I guess all the interfaces need to be fixed (since some are working - see edit trace).

No, the data is not correctly stored.

Traces are recorded in two ways on the server. Firstly, they're imported into a table of GPX points. To the best of my knowledge, this truncates timestamps to the second. Potlatch uses this table when you press 'g', because the table is indexed so you can query it by bbox.

Secondly, the original .gpx file is also saved on the server, unchanged (except potentially for .gz compression to save space). Potlatch uses these files when you click 'edit' by a trace. Consequently it is able to retrieve the subsecond timestamp information.

OK, but why are entire long sections missing (see trace 322818 in the snapshot)?

Don't know off-hand. If you can extract the right bit of the trace I'll have a look at it, but I'm on a mobile connection right now and can't download a 5Mb GPX.

I think that someone used an unappropriately sized hammer to fix a issue which doesn't seem hard to fix otherwise (find identically timed points and eliminate/ignore them).

This sounds fixable, can it be done?

No, for the above reason.

I think the best solution is to actually allow such high resolutions, not allow points with the same timestamp

Maybe so, but you're talking to the wrong guy here. Potlatch is doing the best it can with the data that exists in the database. I don't know anything about the GPX importer, so I suggest you re-enter/refile as a ticket against the API.

Author: eddyp
[Added to the original trac issue at 6.37pm, Wednesday, 18th March 2009]

Replying to [comment:3 Richard]:

Replying to [comment:2 eddyp]:

Replying to [comment:1 Richard]:

The OSM database doesn't import sub-second timestamps on GPS tracks: anything after the second will be truncated.

That is really a bad decision. The tracks which we have from Norc are sampled at 5 samples/second and they have corrections (dgps) so they are really accurate.

Is there a way to fix this now, or do the tables need to altered? The downloaded trace contains the subseconds, so the data is correctly stored, so I guess all the interfaces need to be fixed (since some are working - see edit trace).

No, the data is not correctly stored.

Traces are recorded in two ways on the server. Firstly, they're imported into a table of GPX points. To the best of my knowledge, this truncates timestamps to the second. Potlatch uses this table when you press 'g', because the table is indexed so you can query it by bbox.

Secondly, the original .gpx file is also saved on the server, unchanged (except potentially for .gz compression to save space). Potlatch uses these files when you click 'edit' by a trace. Consequently it is able to retrieve the subsecond timestamp information.

Is it possible to get the GPX ids via the indexed table then fetch the complete gpx-es? Would that be too slow?

OK, but why are entire long sections missing (see trace 322818 in the snapshot)?

Don't know off-hand. If you can extract the right bit of the trace I'll have a look at it, but I'm on a mobile connection right now and can't download a 5Mb GPX.

I just tried, Firefox went into an IO spiral.

I think that someone used an unappropriately sized hammer to fix a issue which doesn't seem hard to fix otherwise (find identically timed points and eliminate/ignore them).

This sounds fixable, can it be done?

No, for the above reason.

Is the table limited that way? I mean are the data types for the time stamps not fit to store the information, if that limitation were to e removed?

I think the best solution is to actually allow such high resolutions, not allow points with the same timestamp

Maybe so, but you're talking to the wrong guy here. Potlatch is doing the best it can with the data that exists in the database. I don't know anything about the GPX importer, so I suggest you re-enter/refile as a ticket against the API.

Yes, indeed, taking into account that norcTracks is the account with the second most number of tracks (after LostTracks) which are VERY precise I guess it would make sense to try to fix that in the API, or at least allow a reimport (from the .gz-ed files) without the no-subsecond filter.

Author: eddyp
[Added to the original trac issue at 7.15pm, Wednesday, 18th March 2009]

Submitted bug against the API:

http://trac.openstreetmap.org/ticket/1657

Author: Richard
[Added to the original trac issue at 12.29pm, Monday, 11th May 2009]

Now that we have Postgres, I believe Potlatch itself will order correctly by partial timestamps, so we can close this ticket pending any change or otherwise in the API.

Author: tom[at]compton.nu
[Added to the original trac issue at 1.14pm, Monday, 11th May 2009]

Note that only tracks imported since the move to API 0.6 will have fractional timestamps in the points table.