Skip to content
This repository has been archived by the owner on Jul 24, 2021. It is now read-only.

Virtual nodes make Download more exponentially slower #2728

Closed
openstreetmap-trac opened this issue Jul 23, 2021 · 4 comments
Closed

Virtual nodes make Download more exponentially slower #2728

openstreetmap-trac opened this issue Jul 23, 2021 · 4 comments

Comments

@openstreetmap-trac
Copy link

Reporter: Trav
[Submitted to the original trac issue database at 9.55am, Wednesday, 17th February 2010]

I'm not sure whether this is something that can/should be fixed in Meraartor or is actually really just a Qt bug, but when virtual nodes are enabled the download more function quickly gets exponentially slower as the number of features on the map increases.

From a UI perspective it shows as an inordinate amount of time at the "Parsing XML" step (it quickly becomes minutes even for downloads of only a few hundred metres square, turning off the virtual nodes feature returns the performance to normal.

Running callgrind indicates that the main culprit is QObjectPrivate::removePendingChildInsertedEvents(QObject*) which seems to be triggered by removing virtual nodes in Way::updateVirtuals().

I found this Qt bug report which appears to be related, although despite the claim of the problem being introduced in 4.6 GA I seem to be able to replicate the poor performance of destruction in Qt 4.3.4 (fedora 7 i386) as well as Qt 4.6.2 (Fedora 12 x86_64)
http://bugreports.qt.nokia.com/browse/QTBUG-6546

This is all using a more or less current SVN build.

@openstreetmap-trac
Copy link
Author

Author: Koying
[Added to the original trac issue at 1.26am, Thursday, 18th February 2010]

Could you give an order of magnitude (in number of features already in the document) before it becomes unbearable, please.

Just tried on Windows. Although it effectively becomes slower, which should be investigated, a download on a 50k features document do not seem unbearable.

OTOH, my box has 4Gb, and Merkaartor now uses 500Mb (with a 60Mb baseline at startup). On a box with less RAM, I can imagine it could become VERY slow because of malloc's... Take into account that the virtual nodes are not counted in the info box, so 50k displayed might really be 75k allocated...

Maybe memory consumption should be checked...

@openstreetmap-trac
Copy link
Author

Author: Trav
[Added to the original trac issue at 2.13am, Thursday, 18th February 2010]

I've collected some very rough timings for download more of just the stage displayed as "Parsing XML" These are on a Core 2 Duo 1.83Ghz w/ 4Gb of RAM (running Fedora 12 64bit).

- - "XML Parsing" time
0K - 5K ~1 sec
5K - 10K ~4 sec
10K - 20K ~10 sec
20K - 30K ~11 sec
...
35K - 40K ~20 sec
40K - 50K ~33 sec
50K - 56K ~41 sec

At 56K features the memory usage is reported as 605M virtual, 132M rss, 22M shared, but I have plenty of RAM free (~2.2G available).

As for when it becomes unbearable, obviously that's subjective, but it certainly starts getting annoying at the 30-40K features range (20 seconds+).

@openstreetmap-trac
Copy link
Author

Author: Trav
[Added to the original trac issue at 2.14am, Thursday, 18th February 2010]

Sorry here's slightly more readable list:

<Features Before DL> - <Features After DL> - "XML Parsing" time
 0K -  5K  ~1 sec
 5K - 10K  ~4 sec
10K - 20K ~10 sec
20K - 30K ~11 sec
...
35K - 40K ~20 sec
40K - 50K ~33 sec
50K - 56K ~41 sec

@openstreetmap-trac
Copy link
Author

Author: Koying
[Added to the original trac issue at 11.01am, Sunday, 7th March 2010]

Should be fixed in trunk and 0.15-fixes

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant