Skip to content
This repository has been archived by the owner on Jul 24, 2021. It is now read-only.

osmosis 0.45 read-pgsql error: java.lang.ClassCastException: org.postgresql.util.PGobject cannot be cast to java.util.Map: #5404

Closed
openstreetmap-trac opened this issue Jul 23, 2021 · 9 comments

Comments

@openstreetmap-trac
Copy link

Reporter: Piskvor
[Submitted to the original trac issue database at 7.29pm, Sunday, 12th June 2016]

On a Ubuntu 12.04 box, with Oracle Java 8 (1.8.0_91-b14, 64-bit), I have a local copy of Czech Republic OSM data, updated from http://download.geofabrik.de/europe/czech-republic-updates via --rri and stored in pgSQL 9.1 database; have been successfully building maps from that (i.e. this part works).

Config:

baseUrl=http://download.geofabrik.de/europe/czech-republic-updates
maxInterval=1800

Sequence state as of now:

#Tue Jun 14 08:20:38 CEST 2016
sequenceNumber=1186
timestamp=2016-06-12T20\:29\:02Z

Updating through this command:

osmosis --rri workingDirectory=/home/foobar/.osmosis-minutely/ --sc --wpc user="osmosis" database="osmosis" password="hunter2"

A few days ago, I have updated osmosis from 0.44.1 to 0.45; built a few maps in the meantime, all worked. Today, after the rri update, I started getting a ClassCastException when reading the database, after about 2 real-minutes of runtime (for a task which usually finishes under a minute). When using 0.44.1-SNAPSHOT (built from commit 90770d38e452678ac672a43d0182432b46925e0a), I never get the exception; when using 0.45, I always get it. The maps generated using 0.44.1 seem to be working correctly.

osmosis --read-pgsql database=osmosis user=osmosis --dataset-bounding-box bottom=49.79 top=50.33 left=14.07 right=14.86 completeWays=no --buffer --write-xml file=./praha-extract.osm
Jun 15, 2016 9:31:56 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.45
Jun 15, 2016 9:31:56 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Jun 15, 2016 9:31:56 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Jun 15, 2016 9:31:56 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
[Jun 15, 2016 9:34:31 AM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-read-pgsql failed
java.lang.ClassCastException: org.postgresql.util.PGobject cannot be cast to java.util.Map
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityRowMapper.mapCommonEntityData(EntityRowMapper.java:81)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.NodeRowMapper.mapRow(NodeRowMapper.java:30)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.NodeRowMapper.mapRow(NodeRowMapper.java:17)
        at org.openstreetmap.osmosis.pgsnapshot.common.RowMapperRowCallbackListener.processRow(RowMapperRowCallbackListener.java:46)
        at org.springframework.jdbc.core.JdbcTemplate$RowCallbackHandlerResultSetExtractor.extractData(JdbcTemplate.java:1566)
        at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:459)
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:404)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:470)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:475)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityDao.getFeaturelessEntity(EntityDao.java:178)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityDao.iterate(EntityDao.java:227)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.PostgreSqlDatasetContext.iterateBoundingBox(PostgreSqlDatasetContext.java:406)
        at org.openstreetmap.osmosis.dataset.v0_6.DatasetBoundingBoxFilter.process(DatasetBoundingBoxFilter.java:80)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.PostgreSqlDatasetReader.run(PostgreSqlDatasetReader.java:53)
        at java.lang.Thread.run(Thread.java:745)

Jun 15, 2016 9:34:31 AM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion

[...errors cascading down the pipeline omitted...]

real    2m35.758s
user    0m1.128s
sys     0m0.116s

Tried a different bounding box, same results: works with 0.44.1, not with 0.45:

osmosis --read-pgsql database=osmosis user=osmosis --dataset-bounding-box bottom=49.00 top=49.30 left=16.40 right=16.90 completeWays=yes --write-xml brno-0.45.osm
Jun 15, 2016 11:08:43 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Osmosis Version 0.45
Jun 15, 2016 11:08:43 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Preparing pipeline.
Jun 15, 2016 11:08:43 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Launching pipeline execution.
Jun 15, 2016 11:08:43 AM org.openstreetmap.osmosis.core.Osmosis run
INFO: Pipeline executing, waiting for completion.
Jun 15, 2016 11:09:05 AM org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager waitForCompletion
SEVERE: Thread for task 1-read-pgsql failed
java.lang.ClassCastException: org.postgresql.util.PGobject cannot be cast to java.util.Map
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityRowMapper.mapCommonEntityData(EntityRowMapper.java:81)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.NodeRowMapper.mapRow(NodeRowMapper.java:30)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.NodeRowMapper.mapRow(NodeRowMapper.java:17)
        at org.openstreetmap.osmosis.pgsnapshot.common.RowMapperRowCallbackListener.processRow(RowMapperRowCallbackListener.java:46)
        at org.springframework.jdbc.core.JdbcTemplate$RowCallbackHandlerResultSetExtractor.extractData(JdbcTemplate.java:1566)
        at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:459)
        at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:404)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:470)
        at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:475)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityDao.getFeaturelessEntity(EntityDao.java:178)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityDao.iterate(EntityDao.java:227)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.PostgreSqlDatasetContext.iterateBoundingBox(PostgreSqlDatasetContext.java:406)
        at org.openstreetmap.osmosis.dataset.v0_6.DatasetBoundingBoxFilter.process(DatasetBoundingBoxFilter.java:80)
        at org.openstreetmap.osmosis.pgsnapshot.v0_6.PostgreSqlDatasetReader.run(PostgreSqlDatasetReader.java:53)
        at java.lang.Thread.run(Thread.java:745)

The equivalent process using a PBF extract of the same data finishes successfully, and the results seem to be functionally identical:

osmosis-0.45 --read-pbf-fast workers=8 file=./czech-republic-latest.osm.pbf --log-progress --bounding-box bottom=49.00 top=49.30 left=16.40 right=16.90 completeWays=yes --write-xml brno-0.45.osm

Files generated via 0.44.1 can be downloaded at http://maps.piskvor.org/ - nothing unusual in the daily updates or in the generated data, as far as I can tell.

@openstreetmap-trac
Copy link
Author

Author: brett[at]bretth.com
[Added to the original trac issue at 11.49pm, Wednesday, 15th June 2016]

Thanks for the report. Surprising error, I'm not sure what's changed in this space. I'll look into it as soon as I can find time.

@openstreetmap-trac
Copy link
Author

Author: brett[at]bretth.com
[Added to the original trac issue at 12.30pm, Tuesday, 21st June 2016]

I can't reproduce this error. Perhaps there is something unusual in the database.

In order to diagnose the problem, can you please make the following code modification, recompile, and try again? It will check the type of the tags field and throw an exception containing the data type received and the entity id being processed. Hopefully it will help us narrow down the problem.

Insert the following code at line 81 of file osmosis-pgsnapshot/src/main/java/org/openstreetmap/osmosis/pgsnapshot/v0_6/impl/EntityRowMapper.java

Object rawDbTags = rs.getObject("tags");
if (!(rawDbTags instanceof Map)) {
	throw new OsmosisRuntimeException("Invalid tags field. EntityId=" + rs.getLong("id")
		+ " TagsType=" + rawDbTags.getClass().getName());
}

@openstreetmap-trac
Copy link
Author

Author: brett[at]bretth.com
[Added to the original trac issue at 1.01pm, Tuesday, 21st June 2016]

From #5407, perhaps this is postgres version related. That defect also occurs on postgres 9.1. I can't reproduce it on postgres 9.3.9.

One difference between 0.44.1 and 0.45 is the postgres JDBC driver version. Are you able to try recompiling with the older postgres JDBC driver?

Modify gradle.properties and modify the following line:

dependencyVersionPostgreSql=9.4-1204-jdbc4

to be

dependencyVersionPostgreSql=9.4-1201-jdbc4

Then recompile (i.e. ./gradlew assemble).

@openstreetmap-trac
Copy link
Author

Author: jjoe64
[Added to the original trac issue at 8.53am, Sunday, 28th August 2016]

I have the same issue.

When reading from pgsql I get that error with current version (0.45 or current git branch). I also tried to change the version of postgres driver like mentioned by brett@ but no effect.

This is my command:

package/bin/osmosis --read-pgsql database=osm user=openstreetmap password='openstreetmap' outPipe.0=pg --dataset-bounding-box inPipe.0=pg left=49.0239 bottom=8.3709 right=49.0389 top=8.4108 outPipe.0=dd --write-xml inPipe.0=dd file=../jaaa.osm

When I checkout the v0.44.1 I works perfectly.

@openstreetmap-trac
Copy link
Author

Author: brett[at]bretth.com
[Added to the original trac issue at 12.52am, Monday, 29th August 2016]

This comment by Jocelyn on ticket 5407 might be the key to your problem:
https://trac.openstreetmap.org/ticket/5407#comment:7

It seems to be related to the postgis version. Osmosis is using a postgis 2 library. Are you still using postgis 1.x?

@openstreetmap-trac
Copy link
Author

Author: sjones925
[Added to the original trac issue at 6.09pm, Wednesday, 8th March 2017]

I have this issue as well with Osmosis 0.45 and --read-pgsql. When I switch to Osmosis 0.44.1, things work as expected, but I lose the postgresSchema option. Here are my versions:

PostgreSQL 9.5.5 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 4.8.2-19ubuntu1) 4.8.2, 64-bit
POSTGIS="2.3.0 r15146" GEOS="3.4.2-CAPI-1.8.2 r3921" PROJ="Rel. 4.8.0, 6 March 2012" GDAL="GDAL 1.10.1, released 2013/08/26" LIBXML="2.9.1" LIBJSON="0.11.99" RASTER

And the stacktrace:

SEVERE: Thread for task 1-read-pgsql failed
java.lang.ClassCastException: org.postgresql.util.PGobject cannot be cast to java.util.Map
	at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityRowMapper.mapCommonEntityData(EntityRowMapper.java:81)
	at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.WayRowMapper.mapRow(WayRowMapper.java:30)
	at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.WayRowMapper.mapRow(WayRowMapper.java:18)
	at org.openstreetmap.osmosis.pgsnapshot.common.RowMapperRowCallbackListener.processRow(RowMapperRowCallbackListener.java:46)
	at org.springframework.jdbc.core.JdbcTemplate$RowCallbackHandlerResultSetExtractor.extractData(JdbcTemplate.java:1566)
	at org.springframework.jdbc.core.JdbcTemplate$1QueryStatementCallback.doInStatement(JdbcTemplate.java:459)
	at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:404)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:470)
	at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:475)
	at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityDao.getFeaturelessEntity(EntityDao.java:178)
	at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityDao.iterate(EntityDao.java:227)
	at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.EntityDao.iterate(EntityDao.java:256)
	at org.openstreetmap.osmosis.pgsnapshot.v0_6.impl.PostgreSqlDatasetContext.iterate(PostgreSqlDatasetContext.java:210)
	at org.openstreetmap.osmosis.dataset.v0_6.DumpDataset.process(DumpDataset.java:48)
	at org.openstreetmap.osmosis.pgsnapshot.v0_6.PostgreSqlDatasetReader.run(PostgreSqlDatasetReader.java:53)
	at java.lang.Thread.run(Thread.java:745)

Mar 08, 2017 8:41:45 PM org.openstreetmap.osmosis.core.Osmosis main
SEVERE: Execution aborted.
org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed.
	at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146)
	at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92)
	at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:330)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:238)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
	at org.codehaus.classworlds.Launcher.main(Launcher.java:47)

@openstreetmap-trac
Copy link
Author

Author: sjones925
[Added to the original trac issue at 11.30pm, Wednesday, 15th March 2017]

If anyone else runs into this, my colleague found that removing lib/default/postgresql-9.1-901-1.jdbc4.jar jar solved this problem. There's another, newer version in the same folder, and keeping that one made the issue go away.

@openstreetmap-trac
Copy link
Author

Author: aviv.rosenberg[at]gmail.com
[Added to the original trac issue at 9.33am, Wednesday, 15th November 2017]

Ran into the same issue with 0.45.
The above comments helped me. After downgrading osmosis to 0.44.1 the problem was resolved.

It seems that in version 0.44.1, an old postgresql client library came back, causing this problem:

[aviv@stark:~/Downloads/osm]
% ll /usr/local/Cellar/osmosis/0.43/libexec/lib/default/postgres*
-rw-rw-r--@ 1 aviv  wheel   538K Apr  2  2013 /usr/local/Cellar/osmosis/0.43/libexec/lib/default/postgresql-9.1-901-1.jdbc4.jar
[aviv@stark:~/Downloads/osm]
% ll /usr/local/Cellar/osmosis/0.44.1/libexec/lib/default/postgres*
-rw-rw-r--  1 aviv  admin   633K Jun  4  2015 /usr/local/Cellar/osmosis/0.44.1/libexec/lib/default/postgresql-9.4-1201-jdbc4.jar
[aviv@stark:~/Downloads/osm]
% ll /usr/local/Cellar/osmosis/0.45/libexec/lib/default/postgres*
-rw-rw-r--  1 aviv  admin   538K May 27  2016 /usr/local/Cellar/osmosis/0.45/libexec/lib/default/postgresql-9.1-901-1.jdbc4.jar
-rw-rw-r--  1 aviv  admin   641K May 27  2016 /usr/local/Cellar/osmosis/0.45/libexec/lib/default/postgresql-9.4-1204-jdbc4.jar

@openstreetmap-trac
Copy link
Author

Author: brett[at]bretth.com
[Added to the original trac issue at 12.29pm, Saturday, 18th November 2017]

This issue should be resolved in 0.46 ... finally. The duplicate postgresql libraries no longer exist.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

1 participant