Ticket #3125 (new enhancement)

Opened 4 years ago

Last modified 3 years ago

Implement trackpoints API call in cgimap

Reported by: amm Owned by: rails-dev@…
Priority: minor Milestone:
Component: api Version:
Keywords: Cc:

Description

Given that cgimap appears to work quite well for the map call and saves quite a few resources on the rails servers, I wanted to extend cgimap to some of the other API calls.

This is an initial proof of concept implementation of the gps trackpoints api call to get some opinions of if this is a viable way.

I think, the attached patch should mostly work, however it is only very lightly tested, as I don't have more than a few gpx tracks in my testing db. There are also a couple of known issues: Formost, it does not yet switch between call types and one has to specify at time of start if it should server map calls, or trackpoints calls. Also the privacy settings of the tracks are possibly not quite correct yet. However, both should be fairly easily fixed.

Doing some initial benchmarking, it appears to again have about a factor of 10 wall clock speedup compared to the rails_port (on a cache hot (nearly empty) database). It shifts from mostly rails cpu dominated to more or less postgresql cpu dominated.

If the trackpoints call is in anyway a bottleneck currently I can obviously not say (that would need some rails log analysis to see where most of the resources are used), but it was the easiest call to implement and can't really harm either.

As said earlier, I am throwing this patch out there for discussion if we should move more of the API into the C backend and what the best approach would be.

Attachments

0001-An-initial-attempt-at-including-the-gps-trackpoints-.patch Download (18.1 KB) - added by amm 4 years ago.
first priliminary patch to include trackpoints into cgimap
0002-Improve-the-initial-trackpoints-api-call.patch Download (8.7 KB) - added by amm 4 years ago.
fixup someof the issues with the first patch

Change History

Changed 4 years ago by amm

first priliminary patch to include trackpoints into cgimap

comment:1 Changed 4 years ago by TomH

I've only had a quick look at the patch, but it looks like you've added a switch that makes it startup in a different mode where all it does is trackpoints queries.

My plan had always been to add some code to the top level loop to inspect the URL and work out which call it was being asked to process so we can just run one set of daemons that can handle anything.

comment:2 Changed 4 years ago by amm

That's correct. But it wasn't intended to stay that way. I remember having a conversation with Matt about how best to handle this, but I can't remember if there was a conclusion and what it was, so I simply did it via a switch for the moment. The code presumably also needs a bit more refactoring to separate out the generic parts from the call specific parts.

comment:3 Changed 4 years ago by amm

Ok, I have now changed it so that it looks at the request URI, so both the map and trackpoints call can be handled by the same daemons.

I think I have also now corrected what I believe is the correct handling of the public / private / trackable / identifiable properties.

So I think this patch now does the right thing now... However, I still haven't tested it thoroughly for a lack of a test db with a reasonable number of traces included.

I have successfully run it through apache bench (100.000 identical requests), so hopefully it at least doesn't leak any memory are have stability issues.

The patch is on top of the first patch, but I can colapse it into a single patch if wished.

Changed 4 years ago by amm

fixup someof the issues with the first patch

comment:4 Changed 4 years ago by TomH

Would it not be easier to fork the git repo and do your work there? Either the master one at  http://git.openstreetmap.org/cgimap.git or zere's one on github.

comment:5 Changed 4 years ago by amm

I have now forked zere's github branch and pushed it there. (  http://github.com/apmon/openstreetmap-cgimap/ )

I have also added an initial implementation of the relations/:id/full call

comment:6 Changed 3 years ago by TomH

  • Owner changed from tom@… to rails-dev@…
Note: See TracTickets for help on using tickets.