Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tiles cache does not work for low zoom #260

Open
adam-boduch opened this issue Sep 3, 2021 · 13 comments
Open

Tiles cache does not work for low zoom #260

adam-boduch opened this issue Sep 3, 2021 · 13 comments

Comments

@adam-boduch
Copy link

I ran render_list to create titles cache.

Meta tiles rendered: Rendered 1398103 tiles in 61960.41 seconds (22.56 tiles/s)
Total tiles rendered: Rendered 89478592 tiles in 61960.41 seconds (1444.13 tiles/s)
Total tiles handled: Rendered 1398103 tiles in 61960.41 seconds (22.56 tiles/s)
*****************************************************
Zoom 01: min:  1.2    avg:  1.2     max:  1.2     over a total of      1.2s in 1 requests
Zoom 02: min: 104.1    avg: 104.1     max: 104.1     over a total of    104.1s in 1 requests
Zoom 03: min: 104.2    avg: 104.2     max: 104.2     over a total of    104.2s in 1 requests
Zoom 04: min:  0.6    avg: 535.4     max: 1605.0     over a total of   1606.2s in 3 requests
Zoom 05: min:  0.2    avg: 121.8     max: 1583.3     over a total of   1826.5s in 15 requests
Zoom 06: min:  0.2    avg: 61.8     max: 1750.2     over a total of   3952.6s in 64 requests
Zoom 07: min:  0.2    avg: 15.6     max: 1294.7     over a total of   3971.6s in 255 requests
Zoom 08: min:  0.2    avg:  6.2     max: 892.9     over a total of   6375.1s in 1024 requests
Zoom 09: min:  0.2    avg:  1.7     max: 599.4     over a total of   7071.1s in 4096 requests
Zoom 10: min:  0.2    avg:  0.6     max: 237.7     over a total of  10385.3s in 16384 requests
Zoom 11: min:  0.2    avg:  0.5     max: 174.2     over a total of  32224.5s in 65536 requests
Zoom 12: min:  0.2    avg:  0.3     max: 122.0     over a total of  91320.5s in 262141 requests
Zoom 13: min:  0.2    avg:  0.3     max: 96.5     over a total of 314768.3s in 1048564 requests

Cache works fine for higher zoom. For lower zoom tiles are being rendered over and over again instead of being read from cache.

[Fri Sep 03 10:42:41.600301 2021] [tile:debug] [pid 19:tid 139994266433280] ./src/mod_tile.c(374): [client 10.0.3.10:55890] tile_state: determined state of ajt 5 4 4 on store 7f52e4000f50: Tile size: 149379, expired: 1 created: 1630391540

As you can see tile was created 3 days ago but still it's expired already.

@stephankn
Copy link
Contributor

Can you share the relevant section of your configuration file? Especially the CacheDuration lines.

@adam-boduch
Copy link
Author

I have mostly default configuration values:

ModTileRenderdSocketAddr renderd 7654
ModTileRequestTimeout 30
ModTileMissingRequestTimeout 3600
ModTileEnableTileThrottling Off
ModTileCacheDurationLowZoom 9 2592000

It's wired because it works fine with bright osm style but not with default one.

@stephankn
Copy link
Contributor

your tile is 267,021 seconds old.

default configuration:

	scfg->cache_duration_dirty = 15 * 60;
	scfg->cache_duration_last_modified_factor = 0.0;
	scfg->cache_duration_max = 7 * 24 * 60 * 60;
	scfg->cache_duration_minimum = 3 * 60 * 60;
	scfg->cache_duration_low_zoom = 6 * 24 * 60 * 60;
	scfg->cache_duration_medium_zoom = 1 * 24 * 60 * 60;

you are overwriting the settings for LowZoom. So for zoom levels 0-9, your minimum cache duration should be quite large.

Can you also share how you handle planet-import-complete and what timestamp this file has? mtime older than this one would also expire tiles.

@adam-boduch
Copy link
Author

/var/lib/mod_tile does not contain file planet-import-complete. Maybe that's the issue here. I will check it out and let you know.

@stephankn
Copy link
Contributor

if planet-import-complete is not found, then everything older than three days (259.200s) is considered expired. This explains why the file above is expired.

@adam-boduch
Copy link
Author

Ok. I tried again. What I did:

  1. I removed ajt folder.
  2. I created planet-import-complete file in /var/lib/mod_tile.
  3. I called render_list -a -m ajt -z 0 -Z 12 -n 24

Results:

Total for all tiles rendered
Meta tiles rendered: Rendered 349520 tiles in 22890.83 seconds (15.27 tiles/s)
Total tiles rendered: Rendered 22369280 tiles in 22890.83 seconds (977.22 tiles/s)
Total tiles handled: Rendered 349528 tiles in 22890.83 seconds (15.27 tiles/s)
*****************************************************
Zoom 00: min: 239.2    avg: 239.2     max: 239.2     over a total of    239.2s in 1 requests
Zoom 01: min: 239.3    avg: 239.3     max: 239.3     over a total of    239.3s in 1 requests
Zoom 02: min: 239.4    avg: 239.4     max: 239.4     over a total of    239.4s in 1 requests
Zoom 03: min: 239.6    avg: 239.6     max: 239.6     over a total of    239.6s in 1 requests
Zoom 04: min:  0.6    avg: 52.4     max: 156.0     over a total of    157.1s in 3 requests
Zoom 05: min:  0.6    avg:  6.6     max: 61.2     over a total of     92.1s in 14 requests
Zoom 06: min:  1.5    avg: 16.3     max: 58.6     over a total of    996.4s in 61 requests
Zoom 07: min:  1.0    avg: 14.4     max: 377.6     over a total of   3665.5s in 254 requests
Zoom 08: min:  0.9    avg: 13.0     max: 1079.7     over a total of  13276.8s in 1023 requests
Zoom 09: min:  0.8    avg:  6.1     max: 656.1     over a total of  24863.8s in 4096 requests
Zoom 10: min:  0.8    avg:  4.0     max: 615.0     over a total of  65183.4s in 16382 requests
Zoom 11: min:  0.6    avg:  1.8     max: 297.2     over a total of 118907.1s in 65536 requests
Zoom 12: min:  0.6    avg:  1.1     max: 128.2     over a total of 299449.9s in 262138 requests

Tiles are not working for zoom 5:

Screenshot_20210920_151114

@SomeoneElseOSM
Copy link
Contributor

"render_list" will make a bunch of requsts for tile rendering, which should appear in syslog. Are there any errors there? For example this morning on a server of mine this happened:

Rendering all tiles from zoom 5 to zoom 5
Rendering all tiles for zoom 5 from (15, 9) to (23, 15)
Waiting for rendering threads to finish

Total for all tiles rendered
Meta tiles rendered: Rendered 1 tiles in 25.03 seconds (0.04 tiles/s)
Total tiles rendered: Rendered 64 tiles in 25.03 seconds (2.56 tiles/s)
Total tiles handled: Rendered 2 tiles in 25.03 seconds (0.08 tiles/s)

and in syslog I see (among other things):

Sep 20 06:42:01 map renderd[1791]: DEBUG: START TILE ajt 5 16-23 8-15, age 3.35 days
Sep 20 06:42:01 map renderd[1791]: Rendering projected coordinates 5 16 8 -> 0.000000|0.000000 10018754.171400|10018754.171400 to a 8 x 8 tile
Sep 20 06:42:26 map renderd[1791]: DEBUG: DONE TILE ajt 5 16-23 8-15 in 25.026 seconds

One of the corresponding tiles is I think https://map.atownsend.org.uk/hot/5/15/10.png .

@openstreetmap-tiles
Copy link

openstreetmap-tiles commented Sep 20, 2021 via email

@SomeoneElseOSM
Copy link
Contributor

@openstreetmap-tiles There's no "--force" parameter specified, so perhaps it thinks those tiles exist already? See here, but beware that if memory serves multiple versions of render_list with slightly different parameters may have existed.

@adam-boduch
Copy link
Author

Yes, there are some errors:

renderd[36]: DEBUG: START TILE ajt 5 16-23 8-15, new metatile
renderd[36]: ERROR: failed to render TILE ajt 5 16-23 8-15
renderd[36]:    reason: Postgis Plugin: server closed the connection unexpectedly
renderd[36]: DEBUG: DONE TILE ajt 5 16-23 8-15 in 7254.504 seconds

So, despite the error, tile was created within 7254 seconds?

In /var/lib/mod_tile there is only a folder ajt/5/0/0/0/16

@SomeoneElseOSM
Copy link
Contributor

Before you start even thinking about trying to render more than one tile at once I'd suggest trying different individual tiles at different zoom levels and look at what appears in syslog to see if the times to render seem "reasonable" (and not indicative of missing indexes or similar).

So, despite the error, tile was created within 7254 seconds?

No, something went horribly wrong and after that all bets are off.

@stephankn
Copy link
Contributor

@adam-boduch Sounds like you have two different issues now. The earlier point was that you had tiles, but mod_tile was requesting a fresh rendering instead of using the cached tiles.

The recent issue points towards a problem with actually rendering tiles. The time does not sound like a magic number, so probably no timeout. Have you checked logs for failures reported by system or postgresql? Check for OOM or similar, as 24 parallel render threads is already a lot.

This might be a generic support issue and no bug in mod_tile/renderd to track here.

@adam-boduch
Copy link
Author

adam-boduch commented Sep 28, 2021

First of all I have to clarify that I have dockerized this project. Everything works under docker swarm on few servers. I'm getting server closed the connection unexpectedly error in PostgreSQL probably because TCP/IP connection between renderd and PostgreSQL is being killed (PostgreSQL and render work on different machines).

Here's a snippet of logs (I called render_list to create cache only for zoom 6):

Sep 28 12:26:10 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 0-7 0-7, age 1.00 days
Sep 28 12:26:10 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 0 0 -> -20037508.342800|15028131.257100 -15028131.257100|20037508.342800 to a 8 x 8 tile
Sep 28 12:26:10 b263ec4f21a7 renderd[90]: Using web mercator projection settings
Sep 28 12:26:10 b263ec4f21a7 renderd[90]: message repeated 8 times: [ Using web mercator projection settings]
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 0-7 0-7 in 0.382 seconds
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/0/0) with protocol version 2 to fd 17
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/0/8) with protocol version 2 to fd 3
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(0), y(8), mime(image/png), options()
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 0-7 8-15, age 1.00 days
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 0 8 -> -20037508.342800|10018754.171400 -15028131.257100|15028131.257100 to a 8 x 8 tile
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 0-7 8-15 in 0.230 seconds
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/0/8) with protocol version 2 to fd 17
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/0/16) with protocol version 2 to fd 3
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(0), y(16), mime(image/png), options()
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 0-7 16-23, age 1.00 days
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 0 16 -> -20037508.342800|5009377.085700 -15028131.257100|10018754.171400 to a 8 x 8 tile
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 0-7 16-23 in 0.250 seconds
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/0/16) with protocol version 2 to fd 17
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/0/24) with protocol version 2 to fd 3
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(0), y(24), mime(image/png), options()
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 0-7 24-31, age 1.00 days
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 0 24 -> -20037508.342800|0.000000 -15028131.257100|5009377.085700 to a 8 x 8 tile
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 0-7 24-31 in 0.218 seconds
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/0/24) with protocol version 2 to fd 17
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/0/32) with protocol version 2 to fd 3
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(0), y(32), mime(image/png), options()
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 0-7 32-39, age 1.00 days
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 0 32 -> -20037508.342800|-5009377.085700 -15028131.257100|0.000000 to a 8 x 8 tile
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 0-7 32-39 in 0.217 seconds
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/0/32) with protocol version 2 to fd 17
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/0/40) with protocol version 2 to fd 3
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(0), y(40), mime(image/png), options()
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 0-7 40-47, age 1.00 days
Sep 28 12:26:11 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 0 40 -> -20037508.342800|-10018754.171400 -15028131.257100|-5009377.085700 to a 8 x 8 tile
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 0-7 40-47 in 0.217 seconds
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/0/40) with protocol version 2 to fd 17
Sep 28 12:26:12 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:12 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/0/48) with protocol version 2 to fd 3
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(0), y(48), mime(image/png), options()
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 0-7 48-55, age 1.00 days
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 0 48 -> -20037508.342800|-15028131.257100 -15028131.257100|-10018754.171400 to a 8 x 8 tile
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 0-7 48-55 in 0.292 seconds
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/0/48) with protocol version 2 to fd 17
Sep 28 12:26:12 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:12 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/0/56) with protocol version 2 to fd 3
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(0), y(56), mime(image/png), options()
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 0-7 56-63, age 1.00 days
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 0 56 -> -20037508.342800|-20037508.342800 -15028131.257100|-15028131.257100 to a 8 x 8 tile
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 0-7 56-63 in 0.265 seconds
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/0/56) with protocol version 2 to fd 17
Sep 28 12:26:12 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:12 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/8/0) with protocol version 2 to fd 3
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(8), y(0), mime(image/png), options()
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 8-15 0-7, age 1.00 days
Sep 28 12:26:12 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 8 0 -> -15028131.257100|15028131.257100 -10018754.171400|20037508.342800 to a 8 x 8 tile
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 8-15 0-7 in 0.225 seconds
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/8/0) with protocol version 2 to fd 17
Sep 28 12:26:13 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:13 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/8/8) with protocol version 2 to fd 3
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(8), y(8), mime(image/png), options()
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 8-15 8-15, age 1.00 days
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 8 8 -> -15028131.257100|10018754.171400 -10018754.171400|15028131.257100 to a 8 x 8 tile
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 8-15 8-15 in 0.293 seconds
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/8/8) with protocol version 2 to fd 17
Sep 28 12:26:13 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:13 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/8/16) with protocol version 2 to fd 3
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(8), y(16), mime(image/png), options()
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 8-15 16-23, age 1.00 days
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 8 16 -> -15028131.257100|5009377.085700 -10018754.171400|10018754.171400 to a 8 x 8 tile
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 8-15 16-23 in 0.231 seconds
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/8/16) with protocol version 2 to fd 17
Sep 28 12:26:13 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:13 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/8/24) with protocol version 2 to fd 3
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(8), y(24), mime(image/png), options()
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 8-15 24-31, age 1.00 days
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 8 24 -> -15028131.257100|0.000000 -10018754.171400|5009377.085700 to a 8 x 8 tile
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 8-15 24-31 in 0.234 seconds
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/8/24) with protocol version 2 to fd 17
Sep 28 12:26:13 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:13 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/8/32) with protocol version 2 to fd 3
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(8), y(32), mime(image/png), options()
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 8-15 32-39, age 1.00 days
Sep 28 12:26:13 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 8 32 -> -15028131.257100|-5009377.085700 -10018754.171400|0.000000 to a 8 x 8 tile
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 8-15 32-39 in 0.219 seconds
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/8/32) with protocol version 2 to fd 17
Sep 28 12:26:14 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:14 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/8/40) with protocol version 2 to fd 3
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(8), y(40), mime(image/png), options()
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 8-15 40-47, age 1.00 days
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 8 40 -> -15028131.257100|-10018754.171400 -10018754.171400|-5009377.085700 to a 8 x 8 tile
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 8-15 40-47 in 0.210 seconds
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/8/40) with protocol version 2 to fd 17
Sep 28 12:26:14 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:14 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/8/48) with protocol version 2 to fd 3
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(8), y(48), mime(image/png), options()
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 8-15 48-55, age 1.00 days
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 8 48 -> -15028131.257100|-15028131.257100 -10018754.171400|-10018754.171400 to a 8 x 8 tile
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 8-15 48-55 in 0.339 seconds
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/8/48) with protocol version 2 to fd 17
Sep 28 12:26:14 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:14 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/8/56) with protocol version 2 to fd 3
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(8), y(56), mime(image/png), options()
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 8-15 56-63, age 1.00 days
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 8 56 -> -15028131.257100|-20037508.342800 -10018754.171400|-15028131.257100 to a 8 x 8 tile
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 8-15 56-63 in 0.246 seconds
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/8/56) with protocol version 2 to fd 17
Sep 28 12:26:14 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:14 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/16/0) with protocol version 2 to fd 3
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(16), y(0), mime(image/png), options()
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 16-23 0-7, age 1.00 days
Sep 28 12:26:14 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 16 0 -> -10018754.171400|15028131.257100 -5009377.085700|20037508.342800 to a 8 x 8 tile
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 16-23 0-7 in 0.235 seconds
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/16/0) with protocol version 2 to fd 17
Sep 28 12:26:15 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:15 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/16/8) with protocol version 2 to fd 3
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(16), y(8), mime(image/png), options()
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 16-23 8-15, age 1.00 days
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 16 8 -> -10018754.171400|10018754.171400 -5009377.085700|15028131.257100 to a 8 x 8 tile
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 16-23 8-15 in 0.283 seconds
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/16/8) with protocol version 2 to fd 17
Sep 28 12:26:15 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:15 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/16/16) with protocol version 2 to fd 3
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(16), y(16), mime(image/png), options()
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 16-23 16-23, age 1.00 days
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 16 16 -> -10018754.171400|5009377.085700 -5009377.085700|10018754.171400 to a 8 x 8 tile
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 16-23 16-23 in 0.305 seconds
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/16/16) with protocol version 2 to fd 17
Sep 28 12:26:15 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:15 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/16/24) with protocol version 2 to fd 3
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(16), y(24), mime(image/png), options()
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 16-23 24-31, age 1.00 days
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 16 24 -> -10018754.171400|0.000000 -5009377.085700|5009377.085700 to a 8 x 8 tile
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 16-23 24-31 in 0.260 seconds
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/16/24) with protocol version 2 to fd 17
Sep 28 12:26:15 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:15 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/16/32) with protocol version 2 to fd 3
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(16), y(32), mime(image/png), options()
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 16-23 32-39, age 1.00 days
Sep 28 12:26:15 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 16 32 -> -10018754.171400|-5009377.085700 -5009377.085700|0.000000 to a 8 x 8 tile
Sep 28 12:26:16 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 16-23 32-39 in 0.222 seconds
Sep 28 12:26:16 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/16/32) with protocol version 2 to fd 17
Sep 28 12:26:16 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:16 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/16/40) with protocol version 2 to fd 3
Sep 28 12:26:16 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:16 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(16), y(40), mime(image/png), options()
Sep 28 12:26:16 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 16-23 40-47, age 1.00 days
Sep 28 12:26:16 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 16 40 -> -10018754.171400|-10018754.171400 -5009377.085700|-5009377.085700 to a 8 x 8 tile
Sep 28 12:26:17 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 16-23 40-47 in 1.067 seconds
Sep 28 12:26:17 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/16/40) with protocol version 2 to fd 17
Sep 28 12:26:17 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:17 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/16/48) with protocol version 2 to fd 3
Sep 28 12:26:17 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:17 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(16), y(48), mime(image/png), options()
Sep 28 12:26:17 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 16-23 48-55, age 1.00 days
Sep 28 12:26:17 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 16 48 -> -10018754.171400|-15028131.257100 -5009377.085700|-10018754.171400 to a 8 x 8 tile
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 16-23 48-55 in 1.013 seconds
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/16/48) with protocol version 2 to fd 17
Sep 28 12:26:18 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:18 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/16/56) with protocol version 2 to fd 3
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(16), y(56), mime(image/png), options()
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 16-23 56-63, age 1.00 days
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 16 56 -> -10018754.171400|-20037508.342800 -5009377.085700|-15028131.257100 to a 8 x 8 tile
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 16-23 56-63 in 0.302 seconds
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/16/56) with protocol version 2 to fd 17
Sep 28 12:26:18 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:18 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/24/0) with protocol version 2 to fd 3
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(24), y(0), mime(image/png), options()
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 24-31 0-7, age 1.00 days
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 24 0 -> -5009377.085700|15028131.257100 0.000000|20037508.342800 to a 8 x 8 tile
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 24-31 0-7 in 0.231 seconds
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/24/0) with protocol version 2 to fd 17
Sep 28 12:26:18 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:18 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/24/8) with protocol version 2 to fd 3
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(24), y(8), mime(image/png), options()
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 24-31 8-15, age 1.00 days
Sep 28 12:26:18 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 24 8 -> -5009377.085700|10018754.171400 0.000000|15028131.257100 to a 8 x 8 tile
Sep 28 12:26:19 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 24-31 8-15 in 0.945 seconds
Sep 28 12:26:19 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/24/8) with protocol version 2 to fd 17
Sep 28 12:26:19 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:19 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/24/16) with protocol version 2 to fd 3
Sep 28 12:26:19 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:26:19 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(24), y(16), mime(image/png), options()
Sep 28 12:26:19 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 24-31 16-23, age 1.00 days
Sep 28 12:26:19 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 24 16 -> -5009377.085700|5009377.085700 0.000000|10018754.171400 to a 8 x 8 tile
Sep 28 12:30:00 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 24-31 16-23 in 220.437 seconds
Sep 28 12:30:00 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/24/16) with protocol version 2 to fd 17
Sep 28 12:30:00 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:00 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/24/24) with protocol version 2 to fd 3
Sep 28 12:30:00 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:00 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(24), y(24), mime(image/png), options()
Sep 28 12:30:00 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 24-31 24-31, age 1.00 days
Sep 28 12:30:00 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 24 24 -> -5009377.085700|0.000000 0.000000|5009377.085700 to a 8 x 8 tile
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 24-31 24-31 in 21.439 seconds
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/24/24) with protocol version 2 to fd 17
Sep 28 12:30:21 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:21 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/24/32) with protocol version 2 to fd 3
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(24), y(32), mime(image/png), options()
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 24-31 32-39, age 1.00 days
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 24 32 -> -5009377.085700|-5009377.085700 0.000000|0.000000 to a 8 x 8 tile
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 24-31 32-39 in 0.225 seconds
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/24/32) with protocol version 2 to fd 17
Sep 28 12:30:21 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:21 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/24/40) with protocol version 2 to fd 3
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(24), y(40), mime(image/png), options()
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 24-31 40-47, age 1.00 days
Sep 28 12:30:21 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 24 40 -> -5009377.085700|-10018754.171400 0.000000|-5009377.085700 to a 8 x 8 tile
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 24-31 40-47 in 0.228 seconds
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/24/40) with protocol version 2 to fd 17
Sep 28 12:30:22 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:22 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/24/48) with protocol version 2 to fd 3
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(24), y(48), mime(image/png), options()
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 24-31 48-55, age 1.00 days
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 24 48 -> -5009377.085700|-15028131.257100 0.000000|-10018754.171400 to a 8 x 8 tile
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 24-31 48-55 in 0.304 seconds
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/24/48) with protocol version 2 to fd 17
Sep 28 12:30:22 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:22 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/24/56) with protocol version 2 to fd 3
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(24), y(56), mime(image/png), options()
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 24-31 56-63, age 1.00 days
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 24 56 -> -5009377.085700|-20037508.342800 0.000000|-15028131.257100 to a 8 x 8 tile
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 24-31 56-63 in 0.268 seconds
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/24/56) with protocol version 2 to fd 17
Sep 28 12:30:22 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:22 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/32/0) with protocol version 2 to fd 3
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(32), y(0), mime(image/png), options()
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 32-39 0-7, age 1.00 days
Sep 28 12:30:22 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 32 0 -> 0.000000|15028131.257100 5009377.085700|20037508.342800 to a 8 x 8 tile
Sep 28 12:30:23 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 32-39 0-7 in 0.365 seconds
Sep 28 12:30:23 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/32/0) with protocol version 2 to fd 17
Sep 28 12:30:23 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:23 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/32/8) with protocol version 2 to fd 3
Sep 28 12:30:23 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:23 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(32), y(8), mime(image/png), options()
Sep 28 12:30:23 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 32-39 8-15, age 1.00 days
Sep 28 12:30:23 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 32 8 -> 0.000000|10018754.171400 5009377.085700|15028131.257100 to a 8 x 8 tile
Sep 28 12:30:55 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 32-39 8-15 in 32.249 seconds
Sep 28 12:30:55 b263ec4f21a7 renderd[90]: DEBUG: Sending render cmd(3 ajt 6/32/8) with protocol version 2 to fd 17
Sep 28 12:30:55 b263ec4f21a7 render_list: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:55 b263ec4f21a7 render_list: DEBUG: Sending render cmd(6 ajt 6/32/16) with protocol version 2 to fd 3
Sep 28 12:30:55 b263ec4f21a7 renderd[90]: DEBUG: Got incoming request with protocol version 2
Sep 28 12:30:55 b263ec4f21a7 renderd[90]: DEBUG: Got command RenderBulk fd(17) xml(ajt), z(6), x(32), y(16), mime(image/png), options()
Sep 28 12:30:55 b263ec4f21a7 renderd[90]: DEBUG: START TILE ajt 6 32-39 16-23, new metatile
Sep 28 12:30:55 b263ec4f21a7 renderd[90]: Rendering projected coordinates 6 32 16 -> 0.000000|5009377.085700 5009377.085700|10018754.171400 to a 8 x 8 tile
Sep 28 14:31:10 b263ec4f21a7 renderd[90]:    reason: Postgis Plugin: server closed the connection unexpectedly#012#011This probably means the server terminated abnormally#012#011before or while processing the request.#012in executeQuery Full sql was: 'SELECT ST_AsBinary("way") AS geom,"feature","way_pixels" FROM (SELECT#012    way, way_pixels,#012    COALESCE(wetland, landuse, "natural") AS feature#012  FROM (SELECT#012      way,#012      ('landuse_' || (CASE WHEN landuse IN ('forest', 'farmland', 'residential', 'commercial', 'retail', 'industrial',#012                                            'meadow', 'grass', 'village_green', 'vineyard', 'orchard') THEN landuse END)) AS landuse,#012      ('natural_' || (CASE WHEN "natural" IN ('wood', 'sand', 'scree', 'shingle', 'bare_rock', 'heath', 'grassland', 'scrub') THEN "natural" END)) AS "natural",#012      ('wetland_' || (CASE WHEN "natural" IN ('wetland', 'mud') THEN (CASE WHEN "natural" IN ('mud') THEN "natural" ELSE tags->'wetland' END) END)) AS wetland,#012      way_area/NULLIF(POW(8.73566e+06*0.001*0.28,2),0) AS way_pixels,#012      way_area#012    FROM planet_osm_polygon#012    WHERE (landuse IN ('forest', 'farmland', 'residential', 'commercial', 'retail', 'industrial', 'meadow', 'grass', 'village_green', 'vineyard', 'orchard')#012      OR "natural" IN ('wood', 'wetland', 'mud', 'sand', 'scree', 'shingle', 'bare_rock', 'heath', 'grassland', 'scrub'))#012      AND way_area > 0.01*2445.98::real*2445.98::real#012      AND building IS NULL#012  ) AS features#012  ORDER BY way_area DESC, feature#012) AS landcover_low_zoom WHERE "way" && ST_SetSRID('BOX3D(-313086.0678562508 4696291.017843749,5322463.153556248 10331840.23925625)'::box3d, 3857)'
Sep 28 14:31:10 b263ec4f21a7 renderd[90]: DEBUG: DONE TILE ajt 6 32-39 16-23 in 7214.768 seconds

As you can see at the beginning everything goes smoothly. It takes longer to build a tile over time.

There are some long running queries:

  pid  |    duration     |                                                                                            query                                                              
                              | state  
-------+-----------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------
------------------------------+--------
 21986 | 01:49:09.696318 | SELECT ST_AsBinary("way") AS geom,"bridge","stylegroup","tunnel","type" FROM ( SELECT way, COALESCE(highway, railway) AS type, 0 AS tunnel, 0 AS bridge, acces
s, 'fill' AS render,         +| idle
       |                 |   CASE                                                                                                                                                        
                             +| 
       |                 |     WHEN highway IN ('motorway', 'trunk') THEN 'motorway'                                                                                                     
                             +| 
       |                 |     WHEN highway IN ('primary', 'secondary') THEN 'mainroad'                                                                                                  
                             +| 
       |                 |     WHEN highway IN ('motorway_link', 'trunk_link', 'primary_link', 'secondary_link', 'tertiary', 'tertiary_link', 'residential', 'unclassified', 'road', 'liv
ing_street') THEN 'minorroad'+| 
       |                 |     WHEN highway IN ('service', 'track') THEN 'service'                                                                                                       
                             +| 
       |                 |     WHEN highway IN ('path', 'cycleway', 'footway', 'pedestrian', 'steps', 'bridleway') THEN 'noauto'                                                         
                             +| 
       |                 |     WHEN railway IN ('light_rail', 'subway', 'narrow_gauge', 'rail', 'tram') THEN 'railway'                                                                   
                             +| 
       |                 |     ELSE 'other' END AS stylegroup                                                                                                                            
                             +| 
       |                 |   FROM planet_osm_line                                                                                                                                        
                             +| 
       |                 |   WHERE (highway IS NOT NULL OR railway IS NOT NULL)                                                                                                          
                             +| 
       |                 |     AND (tunnel IS NULL OR tunnel = 'no')                                                                                                                     
                             +| 
       |                 |     AND (bridge IS NULL OR bridge = 'no')                                                                                                                     
                             +| 
       |                 |   ORDER BY z_order                                                                                                                                            
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(1807429.970832247 66602                                                                                            
                              | 
 22209 | 00:48:49.275803 | SELECT ST_AsBinary("way") AS geom,"type" FROM ( SELECT way, "building" AS type                                                                                
                             +| idle
       |                 |   FROM planet_osm_polygon                                                                                                                                     
                             +| 
       |                 |   WHERE building NOT IN ('', '0','false', 'no')                                                                                                               
                             +| 
       |                 |   ORDER BY ST_YMin(ST_Envelope(way)) DESC                                                                                                                     
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(-118630.2678986581 5927844.41757517,-96616.40375251533 5949858.281721313)'::box3d, 3857)                           
                              | 
 22242 | 01:08:36.557961 | SELECT ST_AsBinary("way") AS geom,"type" FROM ( SELECT way, way_area AS area, COALESCE(landuse, leisure, "natural", highway, amenity, tourism) AS type        
                             +| idle
       |                 |   FROM planet_osm_polygon                                                                                                                                     
                             +| 
       |                 |   ORDER BY way_area DESC                                                                                                                                      
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(1893115.879539977 6628542.655865757,1894491.746049111 6629918.522374891)'::box3d, 3857)                            
                              | 
 22083 | 01:21:47.677214 | select tablename, indexname from pg_indexes where schemaname = 'public' order by indexname;                                                                   
                              | idle
 21216 | 00:48:48.876718 | SELECT ST_AsBinary("way") AS geom,"name","type" FROM ( SELECT way, waterway AS type, name                                                                     
                             +| idle
       |                 |   FROM planet_osm_line                                                                                                                                        
                             +| 
       |                 |   WHERE waterway IN ('canal', 'river', 'stream')                                                                                                              
                             +| 
       |                 |     AND name IS NOT NULL                                                                                                                                      
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(-118630.2678986581 5927844.41757517,-96616.40375251533 5949858.281721313)'::box3d, 3857)                           
                              | 
 21217 | 00:48:48.801337 | SELECT ST_AsBinary("way") AS geom,"name","type" FROM ( SELECT way, waterway AS type, name                                                                     
                             +| idle
       |                 |   FROM planet_osm_line                                                                                                                                        
                             +| 
       |                 |   WHERE waterway IN ('canal', 'river', 'stream')                                                                                                              
                             +| 
       |                 |     AND name IS NOT NULL                                                                                                                                      
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(-118630.2678986581 5927844.41757517,-96616.40375251533 5949858.281721313)'::box3d, 3857)                           
                              | 
 21988 | 01:49:09.678925 | SELECT ST_AsBinary("way") AS geom FROM ( SELECT way                                                                                                           
                             +| idle
       |                 |   FROM planet_osm_point                                                                                                                                       
                             +| 
       |                 |   WHERE highway = 'turning_circle'                                                                                                                            
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(1807429.970832247 6660264.02260412,1810181.703850516 6663015.755622392)'::box3d, 3857)                             
                              | 
 22210 | 01:08:36.596247 | SELECT ST_AsBinary("way") AS geom,"type" FROM ( SELECT way, waterway AS type                                                                                  
                             +| idle
       |                 |   FROM planet_osm_line                                                                                                                                        
                             +| 
       |                 |   WHERE waterway IN ('river', 'canal', 'stream', 'ditch', 'drain')                                                                                            
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(1891892.887087412 6628542.655865757,1893268.753596548 6629918.522374891)'::box3d, 3857)                            
                              | 
 27135 | 00:10:36.95236  | SELECT ST_AsBinary("way") AS geom FROM ( SELECT way, railway AS type, name, z_order                                                                           
                             +| idle
       |                 |   FROM planet_osm_point                                                                                                                                       
                             +| 
       |                 |   WHERE railway = 'station'                                                                                                                                   
                             +| 
       |                 |   ORDER BY z_order NULLS LAST                                                                                                                                 
                             +| 
       |                 | ) AS DATA WHERE "way" && ST_SetSRID('BOX3D(1582475.796588853 5699068.391917515,1583851.663097989 5700444.258426649)'::box3d, 3857)                            
                              | 
 22211 | 00:48:49.186898 | SELECT ST_AsBinary("way") AS geom,"type" FROM ( SELECT way, "building" AS type                                                                                
                             +| idle
       |                 |   FROM planet_osm_polygon                                                                                                                                     
                             +| 
       |                 |   WHERE building NOT IN ('', '0','false', 'no')                                                                                                               
                             +| 
       |                 |   ORDER BY ST_YMin(ST_Envelope(way)) DESC                                                                                                                     
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(-118630.2678986581 5927844.41757517,-96616.40375251533 5949858.281721313)'::box3d, 3857)                           
                              | 
 22212 | 01:08:36.558465 | SELECT ST_AsBinary("way") AS geom,"type" FROM ( SELECT way, way_area AS area, COALESCE(landuse, leisure, "natural", highway, amenity, tourism) AS type        
                             +| idle
       |                 |   FROM planet_osm_polygon                                                                                                                                     
                             +| 
       |                 |   ORDER BY way_area DESC                                                                                                                                      
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(1891892.887087412 6629765.648318321,1893268.753596548 6631141.514827454)'::box3d, 3857)                            
                              | 
 27131 | 00:08:38.976141 | SELECT ST_AsBinary("way") AS geom,"feature","way_pixels" FROM (SELECT                                                                                         
                             +| active
       |                 |     way, way_pixels,                                                                                                                                          
                             +| 
       |                 |     COALESCE(wetland, landuse, "natural") AS feature                                                                                                          
                             +| 
       |                 |   FROM (SELECT                                                                                                                                                
                             +| 
       |                 |       way,                                                                                                                                                    
                             +| 
       |                 |       ('landuse_' || (CASE WHEN landuse IN ('forest', 'farmland', 'residential', 'commercial', 'retail', 'industrial',                                        
                             +| 
       |                 |                                             'meadow', 'grass', 'village_green', 'vineyard', 'orchard') THEN landuse END)) AS landuse,                         
                             +| 
       |                 |       ('natural_' || (CASE WHEN "natural" IN ('wood', 'sand', 'scree', 'shingle', 'bare_rock', 'heath', 'grassland', 'scrub') THEN "natural" END)) AS "natural
",                           +| 
       |                 |       ('wetland_' || (CASE WHEN "natural" IN ('wetland', 'mud') THEN (CASE WHEN "natural" IN ('mud') THEN "natural" ELSE tags->'wetland' END) END)) AS wetland
,                            +| 
       |                 |       way_area/NULLIF(POW(8.73566e+06*0.001*0.28,2),0) AS way_pixels,                                                                                         
                             +| 
       |                 |       way_area                                                                                                                                                
                             +| 
       |                 |     FROM planet_osm_polygon                                                                                                                                   
                             +| 
       |                 |     WHERE (landuse IN ('forest', 'farmland', 'residential', 'commercial', 'retail', 'industrial', 'meadow', 'grass', 'village_green', 'vineyard', 'orchard')  
                             +| 
       |                 |       OR                                                                                                                                                      
                              | 
 22243 | 01:08:36.55782  | SELECT ST_AsBinary("way") AS geom,"type" FROM ( SELECT way, way_area AS area, COALESCE(landuse, leisure, "natural", highway, amenity, tourism) AS type        
                             +| idle
       |                 |   FROM planet_osm_polygon                                                                                                                                     
                             +| 
       |                 |   ORDER BY way_area DESC                                                                                                                                      
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(1891892.887087412 6628542.655865757,1893268.753596548 6629918.522374891)'::box3d, 3857)                            
                              | 
 22394 | 00:51:07.195712 | SELECT ST_AsBinary("way") AS geom,"bridge","render","stylegroup","tunnel","type" FROM ( SELECT way, COALESCE(highway, railway) AS type, 1 AS bridge, access, r
ender, layer, 0 as tunnel,   +| idle
       |                 |   CASE                                                                                                                                                        
                             +| 
       |                 |     WHEN highway IN ('motorway', 'trunk') THEN 'motorway'                                                                                                     
                             +| 
       |                 |     WHEN highway IN ('primary', 'secondary') THEN 'mainroad'                                                                                                  
                             +| 
       |                 |     WHEN highway IN ('motorway_link', 'trunk_link', 'primary_link', 'secondary_link', 'tertiary', 'tertiary_link', 'residential', 'unclassified', 'road', 'liv
ing_street') THEN 'minorroad'+| 
       |                 |     WHEN highway IN ('service', 'track') THEN 'service'                                                                                                       
                             +| 
       |                 |     WHEN highway IN ('path', 'cycleway', 'footway', 'pedestrian', 'steps', 'bridleway') THEN 'noauto'                                                         
                             +| 
       |                 |     WHEN railway IN ('light_rail', 'subway', 'narrow_gauge', 'rail', 'tram') THEN 'railway'                                                                   
                             +| 
       |                 |     ELSE 'other' END AS stylegroup                                                                                                                            
                             +| 
       |                 |   FROM (                                                                                                                                                      
                             +| 
       |                 |     SELECT *, '1_outline' AS render FROM planet_osm_line                                                                                                      
                             +| 
       |                 |     WHERE bridge NOT IN ('', '0', 'no')                                                                                                                       
                             +| 
       |                 |     UNION ALL                                                                                                                                                 
                             +| 
       |                 |     SELECT *, '2_line' AS render FROM planet_osm_line                                                                                                         
                             +| 
       |                 |     WHERE bridge NOT IN ('', '0', 'no')                                                                                                                       
                             +| 
       |                 |     UNION ALL                                                                                                                                                 
                             +| 
       |                 |     SELECT                                                                                                                                                    
                              | 
 27147 | 00:10:37.554952 | SELECT ST_AsBinary("way") AS geom,"type" FROM ( SELECT way, way_area AS area, COALESCE(landuse, leisure, "natural", highway, amenity, tourism) AS type        
                             +| idle
       |                 |   FROM planet_osm_polygon                                                                                                                                     
                             +| 
       |                 |   ORDER BY way_area DESC                                                                                                                                      
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(1582475.796588853 5699068.391917515,1583851.663097989 5700444.258426649)'::box3d, 3857)                            
                              | 
 27148 | 00:10:36.975611 | SELECT ST_AsBinary("way") AS geom,"type" FROM ( SELECT way, way_area AS area, COALESCE(leisure) AS type                                                       
                             +| idle
       |                 |   FROM planet_osm_polygon                                                                                                                                     
                             +| 
       |                 |   WHERE leisure = 'nature_reserve'                                                                                                                            
                             +| 
       |                 |   ORDER BY way_area DESC                                                                                                                                      
                             +| 
       |                 | ) AS data WHERE "way" && ST_SetSRID('BOX3D(1582475.796588853 5699068.391917515,1583851.663097989 5700444.258426649)'::box3d, 3857)                            
                              | 
 25653 | 00:10:36.783467 | SELECT ST_AsBinary("way") AS geom FROM ( SELECT way, railway AS type, name, z_order                                                                           
                             +| idle
       |                 |   FROM planet_osm_point                                                                                                                                       
                             +| 
       |                 |   WHERE railway = 'station'                                                                                                                                   
                             +| 
       |                 |   ORDER BY z_order NULLS LAST                                                                                                                                 
                             +| 
       |                 | ) AS DATA WHERE "way" && ST_SetSRID('BOX3D(1582475.796588853 5699068.391917515,1583851.663097989 5700444.258426649)'::box3d, 3857)                            
                              | 
 22488 | 00:51:07.180348 | SELECT ST_AsBinary("way") AS geom,"bridge","render","stylegroup","tunnel","type" FROM ( SELECT way, COALESCE(highway, railway) AS type, 1 AS bridge, access, r
ender, layer, 0 as tunnel,   +| idle
       |                 |   CASE                                                                                                                                                        
                             +| 
       |                 |     WHEN highway IN ('motorway', 'trunk') THEN 'motorway'                                                                                                     
                             +| 
       |                 |     WHEN highway IN ('primary', 'secondary') THEN 'mainroad'                                                                                                  
                             +| 
       |                 |     WHEN highway IN ('motorway_link', 'trunk_link', 'primary_link', 'secondary_link', 'tertiary', 'tertiary_link', 'residential', 'unclassified', 'road', 'liv
ing_street') THEN 'minorroad'+| 
       |                 |     WHEN highway IN ('service', 'track') THEN 'service'                                                                                                       
                             +| 
       |                 |     WHEN highway IN ('path', 'cycleway', 'footway', 'pedestrian', 'steps', 'bridleway') THEN 'noauto'                                                         
                             +| 
       |                 |     WHEN railway IN ('light_rail', 'subway', 'narrow_gauge', 'rail', 'tram') THEN 'railway'                                                                   
                             +| 
       |                 |     ELSE 'other' END AS stylegroup                                                                                                                            
                             +| 
       |                 |   FROM (                                                                                                                                                      
                             +| 
       |                 |     SELECT *, '1_outline' AS render FROM planet_osm_line                                                                                                      
                             +| 
       |                 |     WHERE bridge NOT IN ('', '0', 'no')                                                                                                                       
                             +| 
       |                 |     UNION ALL                                                                                                                                                 
                             +| 
       |                 |     SELECT *, '2_line' AS render FROM planet_osm_line                                                                                                         
                             +| 
       |                 |     WHERE bridge NOT IN ('', '0', 'no')                                                                                                                       
                             +| 
       |                 |     UNION ALL                                                                                                                                                 
                             +| 
       |                 |     SELECT                                                                                                                                                    
                              | 
(18 rows)

List of indexes:

 spatial_ref_sys_pkey
 simplified_water_polygons_way_idx
 planet_osm_rels_pkey
 planet_osm_rels_parts_idx
 planet_osm_nodes_pkey
 planet_osm_ways_pkey
 planet_osm_ways_nodes_idx
 planet_osm_roads_way_idx
 planet_osm_roads_osm_id_idx
 planet_osm_roads_admin
 planet_osm_roads_admin_low
 planet_osm_roads_roads_ref
 water_polygons_way_idx
 planet_osm_line_way_idx
 planet_osm_line_osm_id_idx
 planet_osm_line_ferry
 planet_osm_line_name
 planet_osm_line_river
 planet_osm_line_waterway
 icesheet_polygons_way_idx
 planet_osm_point_way_idx
 planet_osm_point_osm_id_idx
 planet_osm_point_place
 icesheet_outlines_way_idx
 planet_osm_polygon_way_idx
 planet_osm_polygon_osm_id_idx
 planet_osm_polygon_admin
 planet_osm_polygon_military
 planet_osm_polygon_name
 planet_osm_polygon_nobuilding
 planet_osm_polygon_water
 planet_osm_polygon_way_area_z10
 planet_osm_polygon_way_area_z6
 external_data_pkey
 ne_110m_admin_0_boundary_lines_land_way_idx

PostgreSQL's configuration:

max_connections = 300
shared_buffers = 32GB
effective_cache_size = 96GB
maintenance_work_mem = 2GB
checkpoint_completion_target = 0.9
wal_buffers = 16MB
default_statistics_target = 100
random_page_cost = 1.1
effective_io_concurrency = 200
work_mem = 27962kB
min_wal_size = 2GB
max_wal_size = 8GB
max_worker_processes = 24
max_parallel_workers_per_gather = 4
max_parallel_workers = 24
max_parallel_maintenance_workers = 4

What am I missing?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants