source: subversion/applications/utils/mod_tile/readme.txt @ 29338

Last change on this file since 29338 was 22612, checked in by Dane Springmeyer, 9 years ago

fix a few typos in mod_tile readme

File size: 6.9 KB
Line 
1mod_tile
2========
3A program to efficiently render and serve map tiles for
4www.openstreetmap.org map using Apache and Mapnik.
5
6Note: This program is very much still in development
7it has numerous hard coded paths and options which need
8to be made user configurable options. You may need to
9change these to fit your local environment.
10
11Requirements
12============
13OSM map data imported into PostgreSQL using osm2pgsql
14Mapnik renderer along with the OSM.xml file and map
15symbols, world_boundaries shapefiles. Apache with
16development headers for APR module development.
17
18iniParser
19=========
20renderd uses the ini config file parser from
21http://ndevilla.free.fr/iniparser/ . To make it easier to
22compile, the full distribution is included in the
23subdirectory "iniparser3.0b". Go there and type "make"
24to generate the library before compiling renderd.
25
26Updating From Previous Version
27==============================
28The new version of mod_tile uses a slightly different
29directory hierarchy from the previous version.  In order
30to preserve the previously built tiles you will need to
31move them to:
32
33/var/lib/mod_tile/Default/[Z]/nnn/nnn/nnn/nnn/nnn.png
34
35You will also need to reconfigure your http configuration
36to use the new apache directives and to create an
37/etc/renderd.conf file. See the example renderd.conf and
38mod_tile.conf for details.
39
40Tile Rendering
41==============
42The rendering is implemented in a multithreaded process
43called renderd which opens a unix socket and listens for
44requests to render tiles. It uses Mapnik to render tiles
45using the rendering rules defined in the configuration file
46/etc/renderd.conf
47
48The render daemon implements a queueing mechanism which
49can render foreground requests (for new tiles being viewed)
50and background requests (updating tiles which have expired).
51The size of the queue and the number of threads is determined
52at compile time, see: render_config.h
53
54Tile serving
55============
56To avoid problems with directories becoming too large the
57files are stored in a different layout to that presented
58by the web server. The tiles are now stored under
59/var/lib/mod_tile/[TileSetName]/[Z]/nnn/nnn/nnn/nnn/nnn.png
60
61Where nnn is derived from a combination of the X and Y
62OSM tile co-ordinates.
63
64Apache serves the files as if they were present
65under "/osm_tiles2/Z/X/Y.png" with the path being
66converted automatically.
67
68An Apache module called mod_tile enhances the regular
69Apache file serving mechanisms to provide:
70
711) Tile expiry. It estimates when the tile is next
72likely to be rendered and adds the appropriate HTTP
73cache expiry headers
74
752) When tiles have expired it requests the rendering
76daemon to render (or re-render) the tile.
77
783) Remapping of the file path to the hashed layout
79
80There is an attempt to make the mod_tile code aware of
81the load on the server so that it backs off the rendering
82if the machine is under heavy load.
83
84Setup
85=====
86Make sure you've read and implemented the things in the
87requirements section. Edit the paths in the source to
88match your local setup. Compile the code with make, and
89then make install (as root, to copy the mod_tile to the
90apache module directory).
91
92Create a new apache config file to load the module,
93e.g.
94
95/etc/httpd/conf.d/mod_tile.conf
96
97See the sample mod_tile.conf for details
98
99Edit /etc/renderd.conf to indicate the location of your
100mapnik style sheet and the uri you wish to use to access
101it.  You may configure up to 10 (by default) mapnik
102stylesheets - simply give each section a unique name and
103enter the uri and style sheet path.
104
105Make sure the /var/lib/mod_tile directory is writeable by
106the user running the renderd process and create a file an
107empty file planet-import-complete in this folder.
108
109Run the rendering daemon 'renderd'
110
111Restart Aapche
112
113Note: SELinux will prevent the mod_tile code from opening
114the unix-socket to the render daemon so must be disabled.
115
116Try loading a tile in your browser, e.g.
117http://localhost/osm_tiles2/0/0/0.png
118
119The render daemon should have produce a message like:
120
121Got incoming connection, fd 7, number 1
122Render fd(7) xml(Default), z(0), x(0), y(0)
123
124The disk should start thrashing as Mapnik tries to pull
125in data for the first time. After a few seconds you'll
126probably see a 404 error. Wait for the disk activity to
127cease and then reload the tile. With a bit of luck you
128should see a tile of the world in your browser window.
129
130If this fails to happen check the http error log.  You can
131increate the level of debugging using the LogLevel apache
132directive.  If no log messages are shown check that you
133are accessing the correct virtual host - the new version
134of mod_tile is only installed on a single host by default.
135To install on multiple hosts either use ServerAlias or
136use the LoadTileConfigFile in each virtual host.
137
138To get a complete slippy map you should install a copy
139of the OpenLayers based OSM slippy map and point this to
140fetch tiles from http://localhost/osm_tiles2
141
142mysql2file
143==========
144This was written to export the existing OSM tiles from
145the Mysql database to the filesystem.
146
147
148Bugs
149====
150Too many hard coded options (need to be come module options or command
151line options to renderd).
152mod_tile uses many non-APR routines. It probably only works in Linux.
153If rendering daemon dies then all queued rendering requests are lost.
154Code has not been thoroughly tested.
155
156Performance
157===========
158The existing tile serving based on Apache + mod_ruby + cat_tile.rb
159+ Mysql manages to serve something in the region of 250 - 500 requests
160per second. Apache + mod_tile manages 2000+ per second. Both these
161figures are for tiles which have already been rendered.
162
163Filesystem Issues
164=================
165The average tile size is currently somewhere in the region of 2.5kB.
166(Based on a 20GB MySQL DB which contains 8M tiles). Typically
167filesystems are not particularly efficient at storing large numbers
168of small files. They often take a minimum of 4kB on the disk.
169
170Unfortunately if you reduce the block size to 1 or 2kB then this also
171has a significant impact on the maximum file system size and number of
172inodes available.
173
174
175**  Note: The issues below have been worked around in the current
176code by using the hashed directory path.
177
178The simple z/x/y.png filesystem layout means that at high zoom levels
179there can be large numbers of files in a single directory
180 
181  Zoom 18 = 2^18 = 256k files in a single directory.
182
183If ext2/3 is being used then you really need to have directory indexing
184enabled.
185
186New ".meta" tile storage
187========================
188The latest code stores each metatile in a single .meta file instead of
189lots of small .png files. This is a more efficient use of disk space
190and inodes. For example, many sea tiles are 103 bytes long. In the old
191scheme a meta tile of blank sea tiles would take 64 inodes of 4kB each,
192a total of 256kB. In the new scheme it needs a single file of about 7kB.
193
194The utility convert_meta can be used to convert a tree of .png files to
195.meta (or back again).
196
197mod_tile has been reworked to integrate more closely with Apache and
198deliver tiles from the .meta files.
Note: See TracBrowser for help on using the repository browser.