source: subversion/applications/utils/export/osm2pgsql/README @ 26711

Last change on this file since 26711 was 26688, checked in by apmon, 8 years ago

[osm2pgsql] Update README

File size: 9.8 KB
Line 
1osm2pgsql
2=========
3Converts OSM planet.osm data to a PostgreSQL database suitable
4for rendering into map tiles by Mapnik.
5
6The format of the database is optimised for ease of rendering
7by mapnik. It may be less suitable for other general purpose
8processing.
9
10For a broader view of the whole map rendering tool chain see
11http://wiki.openstreetmap.org/index.php/Mapnik
12http://wiki.openstreetmap.org/index.php/Osm2pgsql
13http://wiki.openstreetmap.org/index.php/Slippy_Map
14
15You may find that the wiki pages are more up to date than this
16readme and may include answers to issues not mentioned here.
17
18Any questions should be directed at the osm dev list
19http://wiki.openstreetmap.org/index.php/Mailing_lists
20
21Features
22========
23- Converts OSM files to a postgresql DB
24- Conversion of tags to columns is configurable in the style file
25- Able to read .gz, .bz2 and .pbf files directly
26- Can apply diffs to keep the database up to data
27- Support the choice of output projection
28- Configurable table names
29- Gazetteer backend for Nominatim
30  http://wiki.openstreetmap.org/wiki/Nominatim
31- Support for hstore field type to store the complete set of tags in one database
32  field if desired
33
34Source code
35===========
36The latest source code is available in the OSM SVN repository
37and can be downloaded as follows:
38
39$ svn co http://svn.openstreetmap.org/applications/utils/export/osm2pgsql
40
41Build requirements
42==================
43The code is written in C and C++ and relies on the libraries
44below:
45- libxml2    http://xmlsoft.org/
46- geos       http://geos.refractions.net/
47- proj       http://www.remotesensing.org/proj/
48- bzip2      http://www.bzip.org/
49- zlib       http://www.zlib.net/
50- PostgreSQL http://www.postgresql.org/
51- PostGIS    http://postgis.refractions.net/
52
53To make use of the database generated by this tool you will
54probably also want to install:
55- Mapnik from http://mapnik.org/
56
57
58
59Building
60========
61Make sure you have installed the development packages for the
62libraries mentioned in the requirements section and a C and C++
63compiler.
64
65e.g. on Fedora:
66# yum install geos-devel proj-devel postgresql-devel libxml2-devel bzip2-devel gcc-c++
67
68on Debian:
69# aptitude install libxml2-dev libgeos-dev libpq-dev libbz2-dev proj autoconf automake libtool make g++
70
71On most Unix-like systems the program can be compiled by
72running './autogen.sh && ./configure && make'.
73
74Operation
75=========
76You must create a PostgreSQL user and a database with the
77postgis functions enabled. This requires access as the
78database administrator, normally the 'postgres' user.
79
80The default name for this database is 'gis' but this may
81be changed by using the osm2pgsql --database option.
82
83If the <username> matches the unix user id running the import
84and rendering then this allows the PostgreSQL 'ident sameuser'
85authentication to be used which avoids the need to enter a
86password when accessing the database. This is setup by default
87on many Unix installs but does not work on Windows (due to the
88lack of unix sockets).
89
90Some example commands are given below but you may find
91this wiki page has more up to data information:
92http://wiki.openstreetmap.org/wiki/Mapnik/PostGIS
93
94$ sudo -u postgres createuser <username>
95$ sudo -u postgres createdb -E UTF8 -O <username> <dbname>
96$ sudo -u postgres createlang plpgsql <dbname>
97
98Adding the postgis extensions. Note the location of the
99files may vary.
100
101$ sudo -u postgres psql <dbname> < /usr/share/postgresql/8.4/contrib/postgis-1.5/postgis.sql
102$ sudo -u postgres psql <dbname> < /usr/share/postgresql/8.4/contrib/postgis-1.5/spatial_ref_sys.sql
103
104Next we need to give the <username> access to update the postgis
105metadata tables
106
107$ sudo -u postgres psql -d <dbname> -c "ALTER TABLE geometry_columns OWNER TO <username>"
108$ sudo -u postgres psql -d <dbname> -c "ALTER TABLE spatial_ref_sys  OWNER TO <username>"
109
110The 900913 is not normally included with postgis. To add it you
111should run:
112
113$ sudo psql -u postgres psql -d <dbname> -f 900913.sql
114
115If you want to use hstore support then you will also need to enable the PostgreSQL
116hstore-new extension.
117
118$ sudo -u postgres psql <dbname> < /usr/share/postgresql/8.4/contrib/hstore.sql
119
120Now you can run osm2pgsql to import the OSM data.
121This will perform the following actions:
122
1231) Osm2pgsql connects to database and creates the following 4 tables:
124   - planet_osm_point
125   - planet_osm_line
126   - planet_osm_roads
127   - planet_osm_polygon
128The prefix "planet_osm" can be changed with the --prefix option,
129the above is the default.
130
1312) Runs an XML parser on the input file (typically planet.osm)
132 and processes the nodes, ways and relations.
133
1343) If a node has a tag declared in the style file then it is
135 added to planet_osm_point. If it has no such tag then
136 the position is noted, but not added to the database.
137
1384) Ways are read in converted into WKT geometries by using the
139 postitions of the nodes read in earlier. If the tags on the way
140 are listed in the style file then the way will be written into
141 the line or roads tables.
142
1435) If the way has one or more tags marked as 'poloygon' and
144 forms a closed ring then it will be added to the lanet_osm_polygon
145 table.
146
1476) The relations are parsed. Osm2pgsql has special handling for a
148 limited number of types: multipolygon, route, boundary
149 The code will build the appropriate geometries by referencing the
150 members and outputing these into the database.
151
1527) Indexes are added to speed up the queries by Mapnik.
153
154Tuning postgresql
155=================
156
157For an efficient operation of postgresql you will need to tune the config
158parameters of postgresql from its default values. These are set in the
159config file at /etc/postgresql/8.4/main/postgresql.conf
160
161The values you need to set will depend on the hardware you have available,
162but you will likely need to increase the values for the following parameters:
163
164- shared_buffers
165- checkpoint_segments
166- work_mem
167- maintenance_work_mem
168- effective_cache_size
169
170
171A quick note on projections
172===========================
173
174Depending on the command-line switches you can select which projection you
175want the database in. You have three choices:
176
1774326: The standard lat/long coordinates
178900913: The spherical mercator projection, used by TileCache, Google Earth etc.
1793395: The legacy (broken) WGS84 mercator projection
180
181Depending on what you're using one or the other is appropriate. The default
182mapnik style (osm.xml) assumes that the data is stored in 900913 and this
183is the default for osm2pgsql.
184
185Combining the -v and -h switches will tell about the exact definitions of
186the projections.
187
188In case you want to use some completely different projection there is the -E
189option. It will initialise the projection as +init=epsg:<num>. This allows
190you to use any projection recognised by proj4, which is useful if you want
191to make a map in a different projection. These projections are usually
192defined in /usr/share/proj/epsg.
193
194Database Access Examples
195========================
196If you wish to access the data from the database then the
197queries below should give you some hints. Note that these
198examples all use the 'latlong' projection which is not the
199default.
200
201$ psql gis
202gis=> \d
203               List of relations
204 Schema |        Name        | Type  |  Owner
205--------+--------------------+-------+----------
206...
207 public | planet_osm_line    | table | jburgess
208 public | planet_osm_point   | table | jburgess
209 public | planet_osm_polygon | table | jburgess
210 public | planet_osm_roads   | table | jburgess
211...
212
213gis=> \d planet_osm_line
214  Table "public.planet_osm_line"
215  Column   |   Type   | Modifiers
216-----------+----------+-----------
217 osm_id    | integer  |
218 name      | text     |
219 place     | text     |
220 landuse   | text     |
221... [ lots of stuff deleted ] ...
222 way       | geometry | not null
223 z_order   | integer  | default 0
224
225
226Each of the tables contains a subset of the planet.osm file representing
227a particular geometry type
228- Point contains nodes which have interesting tags
229  e.g. place=city, name=London
230 
231- Line contains ways with interesting tags
232  e.g. highway=motorway, ref=M25
233 
234- Polygon contains ways which form an enclosed area
235  e.g. landuse=reservoir
236
237The DB columns are used as follows:
238- osm_id = the planet.osm ID of the node(point) or way(line,polygon)
239- name, place, landuse, ... = the value of the given key, if present on
240the node/way. If the tag is not present, the value is NULL. Only a
241subset of all possible tags are stored in the DB. Only ones rendered in
242the osm.xml are actually interesting to mapnik.
243- way = PostGIS geometry describing the physical layout of the object.
244
245
246Querying specific data requires knowlege of SQL and the OSM key/value
247system, e.g.
248
249gis=> select osm_id,astext(way),name from planet_osm_point where amenity='cinema' limit 5;
250  osm_id  |                  astext                   |        name
251----------+-------------------------------------------+--------------------
252 26236284 | POINT(-79.7160836579093 43.6802306464618) |
253 26206699 | POINT(51.4051989797638 35.7066045032235)  | Cinema Felestin
254 26206700 | POINT(51.3994885141459 35.7058460359352)  | Cinema Asr-e Jadid
255 20979630 | POINT(151.225781789807 -33.8943079539886) | Paris Cinema
256 20979684 | POINT(151.226855394904 -33.8946830511095) | Hoyts
257(5 rows)
258
259Mapnik renders the data in each table by applying the rules in the
260osm.xml file.
261
262
263> How could I get e.g. all highways in a given bounding box?
264
265The 'way' column contains the geo info and is the one which you need to
266use in your WHERE clause. e.g.
267
268gis=> select osm_id,highway,name from planet_osm_line where highway is not null and way && GeomFromText('POLYGON((0 52, 0.1 52, 0.1 52.1, 0 52.1, 0 52))',4326);
269 
270osm_id  |   highway    |       name
271---------+--------------+------------------
272 4273848 | unclassified |
273 3977133 | trunk        | to Royston (tbc)
274 4004841 | trunk        |
275 4019198 | trunk        |
276 4019199 | trunk        |
277 4238966 | unclassified |
278
279
280See the Postgis docs for details, e.g.
281http://postgis.refractions.net/docs/ch04.html
Note: See TracBrowser for help on using the repository browser.