Dynmap-Forge/Fabric

Dynmap-Forge/Fabric

1M Downloads

Issue rendering large map in SQLite

Jamboxman5 opened this issue ยท 7 comments

commented

template is bold
sample data is italicized

Issue Description: *I'm fullrendering my map, and it works completely fine from the start. After letting it generate for a long time the map stops working completely. No images load, it constantly says "Waiting for " and it never fully connects. Thinking maybe my map is too big? I've never had this problem before on the same world and nothing has happened to make the map any bigger other than switching from flatfile to sqlite. My database is currently ~23.5GB in size. Everything works again after clearing the database and restarting the render from the beginning.
*

  • Dynmap Version: core=3.4-828, plugin=3.4-828
  • Server Version: paper 1.19.2 build 272
  • Pastebin of Configuration.txt: https://pastebin.com/SF4mRLQ2
  • Server Host (if applicable): None
  • Pastebin of crashlogs or other relevant logs: None
  • Other Relevant Data/Screenshots: Completely reset my configuration and this issue persisted on completely default settings
  • Steps to Replicate: Render a giant map in sqlite

[ X ] I have looked at all other issues and this is not a duplicate
[ X ] I have been able to replicate this

image

commented

Unlikely we will have much option here - if you have logs showing some errors, that might help, but I suspect we're dealing with one or another of SQLite's limitations (number of pages in the DB, page size, or whatever). It really isn't a good large data DB - just a convenient one. I'll give it a look, in case there is something we could do something about, but really cannot make any promises.

commented

@Jamboxman5 what storage are you running on? HDD vs SSD have a huge performance and accessing time difference

commented

Maybe the problem isn't database size.
I have a 157GB Dynmap-SQLite database, and I'm still rendering and updating maps. It works fine and loads quickly.

Edit:
This database file is placed on the mechanical hard disk.

commented

Might be fragmented to hell.

commented

I'm using a brand new PCIe Gen 4 NVMe SSD, a Western Digital Black 1TB SN770

commented

Weird, fragmentation shouldn't matter, then. What filesystem ?

commented

I think GPT, I used whichever the default was in the Ubuntu server installer

EDIT: Just checked and I believe its actually ext4