source: subversion/applications/utils/mod_tile/readme.txt @ 13125

Last change on this file since 13125 was 12950, checked in by twain, 11 years ago

mod_tile: New apache directives and options for multiple tile sets

File size: 6.6 KB
Line 
1mod_tile
2========
3A program to efficiently render and serve map tiles for
4www.openstreetmap.org map using Apache and Mapnik.
5
6Note: This program is very much still in development
7it has numerous hard coded paths and options which need
8to be made user configurable options. You may need to
9these to fit your local environment.
10
11Requirements
12============
13OSM map data imported into PostgreSQL using osm2pgsql
14Mapnik renderer along with the OSM.xml file and map
15symbols, world_boundaries shapefiles. Apache with
16development headers for APR module development.
17
18Updating From Previous Version
19==============================
20The new version of mod_tile uses a slightly different
21directory heirarchy from the previous version.  In order
22to preserve the previously built tiles you will need to
23move them to:
24
25/var/lib/mod_tile/Default/[Z]/nnn/nnn/nnn/nnn/nnn.png
26
27You will also need to recongiure your http configuration
28to use the new apache directives and to create an
29/etc/renderd.conf file. See the example renderd.conf and
30mod_tile.conf for details.
31
32Tile Rendering
33==============
34The rendering is implemented in a multithreaded process
35called renderd which opens a unix socket and listens for
36requests to render tiles. It uses Mapnik to render tiles
37using the rendering rules defined in the configuration file
38/etc/renderd.conf
39
40The render daemon implements a queueing mechanism which
41can render foreground requests (for new tiles being viewed)
42and background requests (updating tiles which have expired).
43The size of the queue and the number of threads is determined
44at compile time, see: render_config.h
45
46Tile serving
47============
48To avoid problems with directories becoming too large the
49files are stored in a different layout to that presented
50by the web server. The tiles are now stored under
51/var/lib/mod_tile/[TileSetName]/[Z]/nnn/nnn/nnn/nnn/nnn.png
52
53Where nnn is derived from a combination of the X and Y
54OSM tile co-ordinates.
55
56Apache serves the files as if they were present
57under "/osm_tiles2/Z/X/Y.png" with the path being
58converted automatically.
59
60An Apache module called mod_tile enhances the regular
61Apache file serving mechanisms to provide:
62
631) Tile expiry. It estimates when the tile is next
64likely to be rendered and adds the approriate HTTP
65cache expiry headers
66
672) When tiles have expired it requests the rendering
68daemon to render (or re-render) the tile.
69
703) Remapping of the file path to the hashed layout
71
72There is an attempt to make the mod_tile code aware of
73the load on the server so that it backs off the rendering
74if the machine is under heavy load.
75
76Setup
77=====
78Make sure you've read and implemented the things in the
79requirements section. Edit the paths in the source to
80match your local setup. Compile the code with make, and
81then make install (as root, to copy the mod_tile to the
82apache module directory).
83
84Create a new apache config file to load the module,
85e.g.
86
87/etc/httpd/conf.d/mod_tile.conf
88
89See the sample mod_tile.conf for details
90
91Edit /etc/renderd.conf to indicate the location of your
92mapnik style sheet and the uri you wish to use to access
93it.  You may configure up to 10 (by default) mapnik
94stylesheets - simply give each section a unique name and
95enter the uri and style sheet path.
96
97Make sure the /var/lib/mod_tile directory is writeable by
98the user running the renderd process and create a file an
99empty file planet-import-complete in this folder.
100
101Run the rendering daemon 'renderd'
102
103Restart Aapche
104
105Note: SELinux will prevent the mod_tile code from opening
106the unix-socket to the render daemon so must be disabled.
107
108Try loading a tile in your browser, e.g.
109http://localhost/osm_tiles2/0/0/0.png
110
111The render daemon should have produce a message like:
112
113Got incoming connection, fd 7, number 1
114Render fd(7) xml(Default), z(0), x(0), y(0)
115
116The disk should start thrashing as Mapnik tries to pull
117in data for the first time. After a few seconds you'll
118probably see a 404 error. Wait for the disk activity to
119cease and then reload the tile. With a bit of luck you
120should see a tile of the world in your browser window.
121
122If this fails to happen check the http error log.  You can
123increate the level of debuging using the LogLevel apache
124directive.  If no log messages are shown check that you
125are accessing the correct virtual host - the new version
126of mod_tile is only installed on a single host by default.
127To install on multiple hosts either use ServerAlias or
128use the LoadTileConfigFile in each virtual host.
129
130To get a complete slippy map you should install a copy
131of the OpenLayers based OSM slippy map and point this to
132fetch tiles from http://localhost/osm_tiles2
133
134mysql2file
135==========
136This was written to export the existing OSM tiles from
137the Mysql database to the filesystem.
138
139Bugs
140====
141Too many hard coded options (need to be come module options or command
142line options to renderd).
143mod_tile uses many non-APR routines. It probably only works in Linux.
144If rendering daemon dies then all queued rendering requests are lost.
145Code has not been thoroughly tested.
146
147Performance
148===========
149The existing tile serving based on Apache + mod_ruby + cat_tile.rb
150+ Mysql manages to serve something in the region of 250 - 500 requests
151per second. Apache + mod_tile manages 2000+ per second. Both these
152figures are for tiles which have already been rendered.
153
154Filesystem Issues
155=================
156The average tile size is currently somewhere in the region of 2.5kB.
157(Based on a 20GB MySQL DB which contains 8M tiles). Typically
158filesystems are not particularly efficient at storing large numbers
159of small files. They often take a minimum of 4kB on the disk.
160
161Unfortunately if you reduce the block size to 1 or 2kB then this also
162has a significant impact on the maximum file system size and number of
163inodes available.
164
165
166**  Note: The issues below have been worked around in the current
167code by using the hashed directory path.
168
169The simple z/x/y.png filesystem layout means that at high zoom levels
170there can be large numbers of files in a single directory
171 
172  Zoom 18 = 2^18 = 256k files in a single directory.
173
174If ext2/3 is being used then you really need to have directory indexing
175enabled.
176
177New ".meta" tile storage
178========================
179The latest code stores each metatile in a single .meta file instead of
180lots of small .png files. This is a more efficient use of disk space
181and inodes. For example, many sea tiles are 103 bytes long. In the old
182scheme a meta tile of blank sea tiles would take 64 inodes of 4kB each,
183a total of 256kB. In the new scheme it needs a single file of about 7kB.
184
185The utility convert_meta can be used to convert a tree of .png files to
186.meta (or back again).
187
188mod_tile has been reworked to integrate more closely with Apache and
189deliver tiles from the .meta files.
Note: See TracBrowser for help on using the repository browser.