Opened 10 years ago

Closed 10 years ago

#2728 closed defect (fixed)

Virtual nodes make Download more exponentially slower

Reported by: Travers Carter Owned by: Chris Browet
Priority: major Milestone:
Component: merkaartor Version:
Keywords: Cc:


I'm not sure whether this is something that can/should be fixed in Meraartor or is actually really just a Qt bug, but when virtual nodes are enabled the download more function quickly gets exponentially slower as the number of features on the map increases.

From a UI perspective it shows as an inordinate amount of time at the "Parsing XML" step (it quickly becomes minutes even for downloads of only a few hundred metres square, turning off the virtual nodes feature returns the performance to normal.

Running callgrind indicates that the main culprit is QObjectPrivate::removePendingChildInsertedEvents(QObject*) which seems to be triggered by removing virtual nodes in Way::updateVirtuals().

I found this Qt bug report which appears to be related, although despite the claim of the problem being introduced in 4.6 GA I seem to be able to replicate the poor performance of destruction in Qt 4.3.4 (fedora 7 i386) as well as Qt 4.6.2 (Fedora 12 x86_64)

This is all using a more or less current SVN build.

Change History (4)

comment:1 Changed 10 years ago by Chris Browet

Owner: changed from cbro@… to Chris Browet
Status: newassigned

Could you give an order of magnitude (in number of features already in the document) before it becomes unbearable, please.

Just tried on Windows. Although it effectively becomes slower, which should be investigated, a download on a 50k features document do not seem unbearable.

OTOH, my box has 4Gb, and Merkaartor now uses 500Mb (with a 60Mb baseline at startup). On a box with less RAM, I can imagine it could become VERY slow because of malloc's... Take into account that the virtual nodes are not counted in the info box, so 50k displayed might really be 75k allocated...

Maybe memory consumption should be checked...

comment:2 Changed 10 years ago by Travers Carter

I've collected some very rough timings for download more of just the stage displayed as "Parsing XML" These are on a Core 2 Duo 1.83Ghz w/ 4Gb of RAM (running Fedora 12 64bit).

<Features Before DL> - <Features After DL> - "XML Parsing" time 0K - 5K ~1 sec 5K - 10K ~4 sec 10K - 20K ~10 sec 20K - 30K ~11 sec ... 35K - 40K ~20 sec 40K - 50K ~33 sec 50K - 56K ~41 sec

At 56K features the memory usage is reported as 605M virtual, 132M rss, 22M shared, but I have plenty of RAM free (~2.2G available).

As for when it becomes unbearable, obviously that's subjective, but it certainly starts getting annoying at the 30-40K features range (20 seconds+).

comment:3 Changed 10 years ago by Travers Carter

Sorry here's slightly more readable list:

<Features Before DL> - <Features After DL> - "XML Parsing" time
 0K -  5K  ~1 sec
 5K - 10K  ~4 sec
10K - 20K ~10 sec
20K - 30K ~11 sec
35K - 40K ~20 sec
40K - 50K ~33 sec
50K - 56K ~41 sec

comment:4 Changed 10 years ago by Chris Browet

Resolution: fixed
Status: assignedclosed

Should be fixed in trunk and 0.15-fixes

Note: See TracTickets for help on using tickets.