Opened 7 years ago

Closed 7 years ago

#4625 closed defect (fixed)

bbox parameter not working with pbf files

Reported by: stephankn Owned by: jburgess777@…
Priority: critical Milestone:
Component: osm2pgsql Version:
Keywords: Cc:

Description

When using the --bbox parameter during an initial import it is not working with pbf files as input. Using xml files work as expected.

How to reproduce:

Create test DBs:

createdb -E UTF8 -T template0 -O osm testxml
psql -d testxml -c "CREATE EXTENSION postgis;"
psql -d testxml -c "CREATE EXTENSION hstore;"
echo "ALTER TABLE geometry_columns OWNER TO osm; ALTER TABLE spatial_ref_sys OWNER TO osm;"  | psql -d testxml
createdb -E UTF8 -T template0 -O osm testpbf
psql -d testpbf -c "CREATE EXTENSION postgis;"
psql -d testpbf -c "CREATE EXTENSION hstore;"
echo "ALTER TABLE geometry_columns OWNER TO osm; ALTER TABLE spatial_ref_sys OWNER TO osm;"  | psql -d testpbf

Import same data into both DB one time as xml one time as pbf:

osm2pgsql -v --create --slim --cache 800 --hstore-all --database testxml --username osm --style default.style --number-processes 1 --disable-parallel-indexing --bbox  "-1.55139,12.34888,-1.51251,12.38296" ../burkina_faso.osm.bz2
osm2pgsql -v --create --slim --cache 800 --hstore-all --database testpbf --username osm --style default.style --number-processes 1 --disable-parallel-indexing --bbox  "-1.55139,12.34888,-1.51251,12.38296" -r pbf ../burkina_faso.osm.pbf

Statistic of input file:

osmconvert --out-statistics burkina_faso.osm.pbf
[...]
nodes: 218077

Query database:

psql -U osm -d testpbf -c "select count(1) from planet_osm_nodes;"
 count
--------
 218077
(1 row)

psql -U osm -d testxml -c "select count(1) from planet_osm_nodes;"
 count
-------
  5482

Only the database with xml input source respects the bounding box parameter. With pbf as input file all data is imported.

Change History (2)

comment:1 Changed 7 years ago by stephankn

The code in parse-pbf::processOsmDataNodes() does not call node_wanted().

Could it be it's simply missing there? Might it help to check after this line

lon = lon_offset + (node->lon * granularity);

doing the node_wanted call and only then process the node?

comment:2 Changed 7 years ago by amm

Resolution: fixed
Status: newclosed

This should now be fixed in r28868

However, I am not sure this is the best way, or one isn't still better off doing the bbox cut out as a pre-processing step.

The full data will still be in the planet_osm_ways and planet_osm_rels tables and the import will potentially be even slower than doing the full import, as the cache hit ratio will be lower, which will compensate some of the gains during the indexing steps.

Note: See TracTickets for help on using tickets.