source: subversion/applications/utils/mod_tile/readme.txt @ 8647

Last change on this file since 8647 was 6898, checked in by jonb, 12 years ago

mod_tile: Update readme & example Apache config file

File size: 5.4 KB
Line 
1mod_tile
2========
3
4A program to efficiently render and serve map tiles for
5www.openstreetmap.org map using Apache and Mapnik.
6
7Note: This program is very much still in development
8it has numerous hard coded paths and options which need
9to be made user configurable options. You will not
10be able to use this program without modifying these to
11fit your local environment.
12
13Requirements
14============
15OSM map data imported into PostgreSQL using osm2pgsql
16Mapnik renderer along with the OSM.xml file and map
17symbols, world_boundaries shapefiles. Apache with
18development headers for APR module development.
19
20
21Tile Rendering
22==============
23The rendering is implemented in a multithreaded process
24called renderd which opens a unix socket and listens for
25requests to render tiles. It uses Mapnik to render tiles
26using the rendering rules defined in osm-jb-merc.xml.
27
28The render daemon implements a queueing mechanism which
29can render foreground requests (for new tiles being viewed)
30and background requests (updating tiles which have expired)
31
32Tile serving
33============
34To avoid problems with directories becoming too large the
35files are stored in a different layout to that presented
36by the web server. The tiles are now stored under
37/var/www/html/direct/[Z]/nnn/nnn/nnn/nnn/nnn.png
38
39Where nnn is derived from a combination of the X and Y
40OSM tile co-ordinates.
41
42Apache serves the files as if they were present
43under "/osm_tiles2/Z/X/Y.png" wiht the path being
44converted automatically.
45
46An Apache module called mod_tile enhances the regular
47Apache file serving mechanisms to provide:
48
491) Tile expiry. It estimates when the tile is next
50likely to be rendered and adds the approriate HTTP
51cache expiry headers
52
532) When tiles have expired it requests the rendering
54daemon to render (or re-render) the tile.
55
563) Remapping of the file path to the hashed layout
57
58There is an attempt to make the mod_tile code aware of
59the load on the server so that it backs off the rendering
60if the machine is under heavy load.
61
62Setup
63=====
64Make sure you've read and implemented the things in the
65requirements section. Edit the paths in the source to
66match your local setup. Compile the code with make, and
67then make install (as root, to copy the mod_tile to the
68apache module directory).
69
70Create a new apache config file to load the module,
71e.g.
72/etc/httpd/conf.d/mod_tile.conf
73
74--------------
75
76LoadModule tile_module modules/mod_tile.so
77
78--------------
79
80Create the directories:
81 /var/www/html/osm_tiles2/
82 /var/www/html/direct/
83
84Run the rendering daemon 'renderd'
85
86Make sure the osm_tiles2 directory is writeable by the
87user running the renderd process.
88
89Restart Aapche
90
91Note: SELinux will prevent the mod_tile code from opening
92the unix-socket to the render daemon so must be disabled.
93
94Try loading a tile in your browser, e.g.
95http://localhost/osm_tiles2/0/0/0.png
96
97The render daemon should have produce a message like:
98
99Got incoming connection, fd 7, number 1
100Render z(0), x(0), y(0), path(/var/www/html/osm_tiles2/0/0/0.png)
101
102The disk should start thrashing as Mapnik tries to pull
103in data for the first time. After a few seconds you'll
104probably see a 404 error. Wait for the disk activity to
105cease and then reload the tile. With a bit of luck you
106should see a tile of the world in your browser window.
107
108To get a complete slippy map you should install a copy
109of the OpenLayers based OSM slippy map and point this to
110fetch tiles from http://localhost/osm_tiles2
111
112mysql2file
113==========
114This was written to export the existing OSM tiles from
115the Mysql database to the filesystem.
116
117Bugs
118====
119Too many hard coded options (need to be come module options or command
120line options to renderd).
121mod_tile uses many non-APR routines. It probably only works in Linux.
122If rendering daemon dies then all queued rendering requests are lost.
123Code has not been thoroughly tested.
124
125Performance
126===========
127The existing tile serving based on Apache + mod_ruby + cat_tile.rb
128+ Mysql manages to serve something in the region of 250 - 500 requests
129per second. Apache + mod_tile manages 2000+ per second. Both these
130figures are for tiles which have already been rendered.
131
132Filesystem Issues
133=================
134The average tile size is currently somewhere in the region of 2.5kB.
135(Based on a 20GB MySQL DB which contains 8M tiles). Typically
136filesystems are not particularly efficient at storing large numbers
137of small files. They often take a minimum of 4kB on the disk.
138
139Unfortunately if you reduce the block size to 1 or 2kB then this also
140has a significant impact on the maximum file system size and number of
141inodes available.
142
143
144**  Note: The issues below have been worked around in the current
145code by using the hashed directory path.
146
147The simple z/x/y.png filesystem layout means that at high zoom levels
148there can be large numbers of files in a single directory
149 
150  Zoom 18 = 2^18 = 256k files in a single directory.
151
152If ext2/3 is being used then you really need to have directory indexing
153enabled.
154
155New ".meta" tile storage
156========================
157The latest code stores each metatile in a single .meta file instead of
158lots of small .png files. This is a more efficient use of disk space
159and inodes. For example, many sea tiles are 103 bytes long. In the old
160scheme a meta tile of blank sea tiles would take 64 inodes of 4kB each,
161a total of 256kB. In the new scheme it needs a single file of about 7kB.
162
163The utility convert_meta can be used to convert a tree of .png files to
164.meta (or back again).
165
166mod_tile has been reworked to integrate more closely with Apache and
167deliver tiles from the .meta files.
Note: See TracBrowser for help on using the repository browser.