Dynmap-Forge/Fabric

Dynmap-Forge/Fabric

888k Downloads

Webp tiles are empty files and then removed by Dynmap

teanup opened this issue ยท 12 comments

commented

Issue Description: Dynmap creates empty files as the map tiles and removes those files after a few minutes. Looks similar to #3898 but it's not about a database and I'm not using any other file formats. I'm only generating webp files and using a filetree system (with bigmap setting on). Even after a fresh server start with no file in web/tiles, I run into this issue every time.

  • Dynmap Version: v3.5-beta-3 for Spigot/PaperMC
  • Server Version: paper 1.19.4 build 501
  • Pastebin of Configuration.txt: https://pastebin.com/ZARRempC
  • Server Host (if applicable): self-hosted on a debian bullseye machine (amd64 architecture)
  • Other Relevant Data/Screenshots: The minecraft server is running in a docker container but I copied cwebp and dwebp in an accessible directory for Dynmap. They work, are detected by Dynmap with no issue and have the correct permissions for Dynmap to use them. Here is my worlds.txt file : https://pastebin.com/TGyEd4Yj , and here is an example of a template i'm using : https://pastebin.com/vQdMVpM0
  • Steps to Replicate: Fullrender a map with this configuration and template, then list files in web/tiles/world. At first the files in the sub-directories are webp files but with no data in them. After a few minutes, they're gone

[x] I have looked at all other issues and this is not a duplicate
[x] I have been able to replicate this

commented

update, found a way to run ubuntu 22.04 on an amd64 system, and tiles are not being deleted after a few mins after the render completed so far, so maybe it is a docker issue?

commented

I just tested your setup on my windows 11 amd64 system with the applicable cwebp and dwebp versions, and so far it hasn't deleted any files, don't have a linux environment available to test on quickly

commented

Dynmap doesn't delete the files when I'm generating PNGs or JPGs though, and tiles generated with those formats are accurate and not empty files.
I copied cwebp and dwebp into plugins/dynmap/webp/ and the docker user has all permissions on those, and owns them. And I tried executing them on random PNG and WEBP files, and it worked perfectly.
So I really don't get why this issue happens and how could docker even affect the way the files are generated...

commented

Are you sure your cwebp, dwebp, tilepath and webpath are correct? (They have a / in front, which is not the default configured position)

commented

Dynmaps logs that it detects them:

[11:11:50 INFO]: [dynmap] Enabling dynmap v3.5-beta-3-866
[11:11:51 INFO]: [dynmap] Added 19 custom biome mappings
[11:11:51 INFO]: [dynmap] Using LuckPerms 5.4.71 for access control
[11:11:51 INFO]: [dynmap] SkinsRestorer API integration enabled
[11:11:51 INFO]: [dynmap] Found cwebp at /minecraft/plugins/dynmap/webp/cwebp and dwebp at /minecraft/plugins/dynmap/webp/dwebp: webp format enabled
[11:11:51 INFO]: [dynmap] Mod Support processing completed
[11:11:53 INFO]: [dynmap] Loaded 27 shaders.
[11:11:53 INFO]: [dynmap] Loaded 84 perspectives.
[11:11:53 INFO]: [dynmap] Loaded 23 lightings.
[11:11:53 INFO]: [dynmap] Starting enter/exit processing
[11:11:53 INFO]: [dynmap] Finish marker initialization
[11:11:53 INFO]: [dynmap] version 3.5-beta-3-866 is enabled - core version 3.5-beta-3-866
[11:11:53 INFO]: [dynmap] For support, visit our Discord at https://discord.gg/s3rd5qn
[11:11:53 INFO]: [dynmap] For news, visit https://reddit.com/r/Dynmap or follow https://universeodon.com/@dynmap
[11:11:53 INFO]: [dynmap] To report or track bugs, visit https://github.com/webbukkit/dynmap/issues
[11:11:53 INFO]: [dynmap] If you'd like to donate, please visit https://www.patreon.com/dynmap or https://ko-fi.com/michaelprimm
[11:11:53 INFO]: [dynmap] Loaded 3 maps of world 'world'.
[11:11:53 INFO]: [dynmap] Loaded 3 maps of world 'world_nether'.
[11:11:53 INFO]: [dynmap] Loaded 2 maps of world 'world_the_end'.
[11:11:53 INFO]: [dynmap] Enabled

And yes it's just the "relative absolute" path inside the docker container.
/minecraft is the path to the base minecraft directory, from the scripts running in the docker's "view".

commented

and are those cwebp and dwebp tools the ones for your architecture and OS type? just making sure, I assume it is, if so, I don't know whether there is a fix for this.

commented

Actually I just solved it???

OMG that's it...

I installed webp with APT after I saw it was available. And since it worked on some test images, I didn't notice any issue. But for some reason the version from APT is 0.6.1... They really should update it or at least remove the package from the library.

So actually, webp image tiles are supported, my bad :')
You just need to make sure to use the 1.3.0 version...

commented

adding a plus 1

I have almost identical set up to #3898 .

Running in Oracle Cloud with Ampere ARM64 CPU
OS. Ubuntu 22.04
Rootless Docker
Image https://github.com/itzg/docker-minecraft-server
Running mariadb 10.11.2 on another rootless docker container
Have webp binaries and dynmap detects them

Try to run any render job for the world and it shows that its making the tiles but nothing is set in the New Image column, upon cancelling the render job it deletes all the rows in the Tile table.

I've run dynmap with this setup before though different version of dynmap/ubuntu/and older mariadb on arm64 before, not sure what's changed to cause this now.

commented

the above comment is exactly what I experienced when rendering anything other than webp when webp was commited to the database, but then on bare linux arm64 OCI

commented

Actually I just solved it???

Since I was on a linux arm64 device there was no precompiled webp binaries so I compiled mine 2 years ago on version 1.1.0 , I just reran compiling for the latest 1.3.0 webp version and it just started working! I'm guessing the updated libraries are needed for newer versions of ubuntu or dynmap? Unsure but I just got it working by making my own updated binaries to version 1.3.0.

Check your webp versions and try that.

https://developers.google.com/speed/webp/docs/precompiled

https://developers.google.com/speed/webp/docs/compiling

https://storage.googleapis.com/downloads.webmproject.org/releases/webp/index.html

My steps below to compile your own if you can't get 1.3.0 via other means.

$ sudo apt-get update
$ mkdir /opt/webp
$ cd /opt/webp
$ curl -L -O https://storage.googleapis.com/downloads.webmproject.org/releases/webp/libwebp-1.3.0.tar.gz
$ tar -xvzf libwebp-1.3.0.tar.gz
$ cd libwebp-1.3.0
$ sudo apt-get install libjpeg-dev libpng-dev libtiff-dev libgif-dev
$ sudo apt-get install build-essential
$ ./configure
$ sudo make
$ sudo make install 
$ export LD_LIBRARY_PATH="/usr/local/lib/" or export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib
$ echo $LD_LIBRARY_PATH
$ hash -r
commented

hmm I tried this, on the setup server, and webp files are still not commited to the database after a succesfull fullrender of that map: https://dynmap-setup.jurgenmk.nl/?worldname=world_formats&mapname=surfacewebp

commented

oh my, I found that I didn't add the webp tool to path, works now for me too, thank you so much!