Issue Description: In reference to #1796: The issue STILL persists to this day

• Dynmap Version: 3.1-SNAPSHOT-432 (dynmap-HEAD-spigot.jar)

• Server Version: papermc-265 (1.16.4)

• Pastebin of Configuration.txt: https://pastebin.com/CeGFpptg

• Pastebin of normal-flat.txt: https://pastebin.com/KgqZr3kf

• Server Host (if applicable): self hosted

• Pastebin of crashlogs or other relevant logs: no crashes or errors or even warnings

• Other Relevant Data/Screenshots:
  
  

• Steps to Replicate:

1. Create new Server (view distance is 32)
2. fly around a while (in creative)
3. see black bars appear in zoom out render
4. even dynmap fullrender does not fix those

• I have looked at all other issues and this is not a duplicate
• I have been able to replicate this

@BrainStone made a PR #3436 which helps this issue but does not completely fix it (reportedly) If we could get more data on this issue with the latest snapshot published at https://dynmap.us/builds/dynmap/ that would be awesome

More data on this issue:

I'm updated to Dynmap-3.3-beta-5-spigot.jar.
I did the following:

dynmap purgeworld world
dynmap fullrender world

The render process is long, when I access console again, the time passed > 24 hours.
After all render done, I visit the web page, but black tile still exist if I zoom out.
Then, as @Pingger says, change allowReconnect to autoReconnect:

port: 3306
database: dynmap?autoReconnect=true&useSSL=false

Restart server, this time all tiles shows correctly no matter how I zoom, without re-render again.
I need to run the server longer than 12 hours to see if the problem still appears.

Major release of Dynmap is out, including some db changes along the way. Is it confirmed that this is specific to 'jdbc'? Can some folks please confirm and replication this issue? Thanks!

Why is this still a problem, this thread was created 2 years ago and this problem still exists. I have a map that I spent a long time rendering but after it finished I ended up expanding max zoomout. But for some reason, dynmap does not check/re-render only those zoom areas or anything at all. This is a ridiculous problem and I do not want to do a complete fullrender of the map. Can someone help? Thanks!
PS: No I dont use jdbc or anything like that, all the images are stored locally.

Why is this still a problem, this thread was created 2 years ago and this problem still exists. I have a map that I spent a long time rendering but after it finished I ended up expanding max zoomout. But for some reason, dynmap does not check/re-render only those zoom areas or anything at all. This is a ridiculous problem and I do not want to do a complete fullrender of the map. Can someone help? Thanks! PS: No I dont use jdbc or anything like that, all the images are stored locally.

That is simply how dynmap works, it is best to first set the settings and then do a fullrender, zoomout tiles will get rendered slowly by use of the zoomout processing timer in configuration.txt

I've been having this problem for a while too regardless of storage type but I believe I found a fix/avoidance for it.
I generally had several maps being rendered at once, Whether it's different worlds or just different maps of the same world and would experiance the problem with most if not all of them. However when rendering just a single map at a time, I have yet to run into the issue. I noticed in the first screenshots here there 5 different maps for your normal world. Might have only been rendering the one world at a time but very likely that you were rendering all 5 of the maps for that world at once. Try rendering just a single map at a time via "dynmap fullrender <world name>:<map name>"

Bumping this since it seems to be a lot worse now on 1.19, but it may just be bad luck on my part. Any updates on ever being properly fixed for db (sqlite) storage? Alternatively, was the tool mentioned by @Pingger or something similar ever been published? I would really like to cleanup this mess plaguing almost all my maps...

you should have the "new flags" autoReconnect and allowReconnect set in configuration. the next thing you can do is increase the wait_timeout in the db configuration for mysql, sqlite as far as I know has no issues with conntimeout. What also helps to reduce the black zoomout tiles is not restarting while dynmap is rendering, because it sometimes doesn't keep track of what it was rendering with zoomout tiles.

closing this issue due to inactivity, original user did not respond for a while and issue not apparent if not suddenly restarting.

closing this issue due to inactivity, original user did not respond for a while and issue not apparent if not suddenly restarting.

Sometimes stopping/restarting is due to external issues. Is there anything one can do to prevent this issue from happening when you do restart before its done rendering? A way to pause/resume the render manually would be really nice.

closing this issue due to inactivity, original user did not respond for a while and issue not apparent if not suddenly restarting.

Sometimes stopping/restarting is due to external issues. Is there anything one can do to prevent this issue from happening when you do restart before its done rendering? A way to pause/resume the render manually would be really nice.

you can try the command /dynmap pause to pause some types of render (/dynmap pause none to resume)

Awesome! I had no idea those are a thing, thanks! will try that next time i get to do a fullrender.

Would it be possible to submit a feature request to enable that behaviour (or, at the very least. can an option be added to enable such a thing?)

The machine the server is running on utilizes a pair of Xeon 2697 v2's. While providing 48 threads, their modern day performance is comparatively lacking to even single socket machines that exist today.

Running the initial dynmap fullrender takes about 8~ hours to complete on the current world file. Extrapolating out, with roughly 1.5 threads in use (going by reported system load below) it will take approximately 32 times longer, or 256 hours (approximately 10.667 days).

I'm not sure if that's per zoom level or for the entire remainder of the map that load average will be maintained. But hopefully it can help explain why I would like to press for this feature to be altered

yes, zoomout tiles are rendered on the background while the server is "idling" also, yes, you are necroposting as the issue you have isnot an issue and not about this issue above.

I can see this issue got flagged as closed. So hopefully I'm not necroposting (all other issues on this were linked back to this main one)

I've got a completely vanilla PaperMC 1.19.2 instance running with Dynmap as the only plugin

No matter how many times I execute a dynmap fullrender <worldname> it will churn through and eventually complete, however it's only rendered up to a certain zoom in distance. Loading the map initially shows a black screen until I zoom in several times, which due to the size of the server world makes things difficult to navigate.

Is there something on my end I may be missing?

@JurgenKuyper Apologies. #2358 linked to this ticket so I was not sure if it was a bug or feature or not.

Is there any way to have Dynmap render all zoom levels and peg the CPU in doing so instead of waiting for it to be considered 'idle'?

No, as said zoomouts are done in the background and thus not editable, just don't restart while fullrendering or pause all, restart, pause none

I am not sure if this is related - but I have this phenomenon in the iso-map...

Is there any way to fix this temporarily?
@Pingger would you share your small custom-made tool that kills the zoomouts?
or alternatively, can you give any clues on how to proceed here?

I had the same issue when using MySQL, but this was fine using filetree and SQLite

Its a paper specific bug :/

no it is not I can also replicate the exact same issue with a forgeserver and plain spigot

Can you send screenshots of this happening on spigot or bukkit?

Here you go for spigot 1.16.3 (git-Spigot-57bbdd8-dea4138) and dynmap 3.1-SNAPSHOT-432:
Settings:
Default, except:

• viewdistance 32
• level-name set to worldxx
• dynmap port set to 9999
• dynmap storage set to sqlite

4:1 Zoomout:

2:1 zoomout:

Another 2:1 zoomout after a bit of exploring:

Result after another fullrender:

It seems this is specifically an issue with jdbc storages: filetree seems to work fine, while sqlite and mysql produce these broken zoomouts ...
My temporary workaround:
A small custom made tool, that searches for zoomouts, where image is blank and deletes those from the database ... after a while dynmap regenerates those correctly. It seems to be a race-condition between normal rendering and zoom-out processing.

Interesting that it only breaks on jdbc storages. This is good info

My uneducated guess: The issue lies in the "lastUpdate" timestamps. the zoomout version should have the timestamp equal to the latest subtile's timestamp it uses. Currently it is the UTC of the minecraft server after it finishes the zoomout. When during this time a subtile is updated it is not included in the zoomout, but the timestamp of the zoomout makes it look like it is included and thus causing the inconsistencies.
The issue is just more apparent on papermc, because the main server thread has a higher thread priority and starves the dynmap zoomout renderthread

After particularly long periods of not restarting the minecraft server (>12h), I also sometimes get this Error:

[21:48:44 ERROR]: [dynmap] Tile read error - The last packet successfully received from the server was 31,062,641 milliseconds ago.  The last packet sent successfully to the
 server was 31,062,642 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity
 before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this
 problem.

your default flags value is allowReconnect not autoReconnect changing that seems to severely reduce the blackbars on papermc (not fix them entirely)

It seems this is specifically an issue with jdbc storages: filetree seems to work fine, while sqlite and mysql produce these broken zoomouts ...
My temporary workaround:
A small custom made tool, that searches for zoomouts, where image is blank and deletes those from the database ... after a while dynmap regenerates those correctly. It seems to be a race-condition between normal rendering and zoom-out processing.

Care to share this tool?

Experiencing the same problem with MariaDB and paper, and also occasionally getting the Tile read error.

I had the same issue when using MySQL, but this was fine using filetree and SQLite

Having this issue while using SQLite

It seems this is specifically an issue with jdbc storages: filetree seems to work fine, while sqlite and mysql produce these broken zoomouts ...
My temporary workaround:
A small custom made tool, that searches for zoomouts, where image is blank and deletes those from the database ... after a while dynmap regenerates those correctly. It seems to be a race-condition between normal rendering and zoom-out processing.

i too would be interested in such a tool if you are willing to share.