
Laggy response times, and Memory issue
MrShadowDraco opened this issue ยท 3 comments
The only issue that we've run into is that it gets laggy on large plans. I built a pyramid from it with a base size of 34x34. During the plan phase, I could only place 5-7 blocks at a time reliably, and it would take 10-15 seconds to "catch up" (the blocks I placed would disappear, and then reappear, and my inventory would update back to the starting quantity). If we placed more than 10 blocks at a time, we'd lose our connection and have to reconnect to the server. The "lag" from placing blocks affected everyone on the server, though the connection loss only happened to the player placing the blocks.
During the build phase, I put a bunch of the materials in the chest, and it started building fine, but when it started on the Sandstone (primary material for the pyramid sides), it took a long time to place each block (about 30 seconds for each one I think). The end project had about 1100 Sandstone blocks, so not sure if the number of blocks was causing the slowdown, but it didn't seem to affect the blocks that there were smaller totals for.
Logged out of the server for a while to let it do its thing. When I logged back into my server, it had stopped building, even though there were still materials in the chest. The plan was still all there (checked view). Server was throwing "Can't keep up" errors. Decided to restart the server, and the plugin crashed with an "OutOfMemory" error (have 2 gb set for server). Below is the log showing the error:
161 recipes
17 achievements
00:26:39 [INFO] Starting minecraft server version Beta 1.8.1
00:26:39 [INFO] Loading properties
00:26:39 [INFO] Starting Minecraft server on *:25565
00:26:39 [INFO] This server is running Craftbukkit version git-Bukkit-0.0.0-1067
-g6301507-b1185jnks (MC: 1.8.1)
00:26:40 [INFO] CityWorld has been enabled
00:26:40 [INFO] [Safe Creeper] Safe Creeper V0.3.2 Enabled
00:26:40 [INFO] Spout 1.0.6.381 has been initialized
00:26:40 [INFO] Preparing level "myworld"
00:26:40 [INFO] Default game type: 0
00:26:40 [INFO] Preparing start region for level 0 (Seed: 2055055923)
00:26:41 [INFO] Preparing start region for level 1 (Seed: -8095254886256020997)
00:26:41 [INFO] Preparing spawn area: 0%
00:27:28 [SEVERE] Error occurred while enabling BuildingPlanner v0.2.0 (Is it up
to date?): Java heap space
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOfRange(Unknown Source)
at java.lang.String.(Unknown Source)
at java.lang.StringBuilder.toString(Unknown Source)
at org.yaml.snakeyaml.reader.StreamReader.getMark(StreamReader.java:74)
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchValue(ScannerImpl.java:76
7)
at org.yaml.snakeyaml.scanner.ScannerImpl.fetchMoreTokens(ScannerImpl.ja
va:307)
at org.yaml.snakeyaml.scanner.ScannerImpl.checkToken(ScannerImpl.java:18
3)
at org.yaml.snakeyaml.parser.ParserImpl$ParseBlockMappingValue.produce(P
arserImpl.java:592)
at org.yaml.snakeyaml.parser.ParserImpl.peekEvent(ParserImpl.java:163)
at org.yaml.snakeyaml.parser.ParserImpl.checkEvent(ParserImpl.java:148)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:132)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java
:230)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:160)
at org.yaml.snakeyaml.composer.Composer.composeMappingNode(Composer.java
:230)
at org.yaml.snakeyaml.composer.Composer.composeNode(Composer.java:160)
at org.yaml.snakeyaml.composer.Composer.composeDocument(Composer.java:12
2)
at org.yaml.snakeyaml.composer.Composer.getSingleNode(Composer.java:105)
at org.yaml.snakeyaml.constructor.BaseConstructor.getSingleData(BaseCons
tructor.java:124)
at org.yaml.snakeyaml.Yaml.load(Yaml.java:264)
at org.bukkit.util.config.Configuration.load(Configuration.java:82)
at za.dats.bukkit.buildingplanner.model.PlanArea.loadArea(PlanArea.java:
541)
at za.dats.bukkit.buildingplanner.PlanAreaManager.init(PlanAreaManager.j
ava:77)
at za.dats.bukkit.buildingplanner.BuildingPlanner.onEnable(BuildingPlann
er.java:45)
at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:126)
at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader
.java:920)
at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManage
r.java:278)
at org.bukkit.craftbukkit.CraftServer.loadPlugin(CraftServer.java:173)
at org.bukkit.craftbukkit.CraftServer.enablePlugins(CraftServer.java:156
)
at net.minecraft.server.MinecraftServer.e(MinecraftServer.java:297)
at net.minecraft.server.MinecraftServer.a(MinecraftServer.java:284)
at net.minecraft.server.MinecraftServer.init(MinecraftServer.java:152)
at net.minecraft.server.MinecraftServer.run(MinecraftServer.java:348)
00:27:28 [INFO] CraftBukkitUpToDate version 2.6.7 is enabled!
00:27:28 [INFO] CButD: Permission system not detected, defaulting to OP
00:27:28 [INFO] InventorySave by Master24 v0.5 enabled!
00:27:28 [INFO] [Multiverse-Core] - Version 2.0-b252 Enabled - By Rigby and fern
ferret
00:27:28 [INFO] [AllPay] - Version 1.0.3 - using only an item based economy for
Multiverse-Core v2.0-b252
00:27:28 [INFO] [Multiverse-Core] - Multiverse Config -- Loaded
00:27:28 [INFO] [Multiverse-Core] - World Config -- Loaded
00:27:28 [INFO] [Multiverse-Core] Loading World & Settings - 'myworld' - NORMAL
00:27:28 [INFO] Preparing start region for level 2 (Seed: -5243929129785998704)
00:27:29 [INFO] CButD: CraftBukkit is up to Date
00:27:29 [INFO] [Multiverse-Core] Loading World & Settings - 'Skylands' - SKYLAN
DS
00:27:29 [INFO] [Multiverse-Core] Loading World & Settings - 'myworld_nether' -
NETHER
00:27:29 [INFO] [Multiverse-Core] 3 - World(s) loaded.
00:27:29 [INFO] [MultiVerse-NetherPortals] - Version 2.0-b225 Enabled - By Rigby
and fernferret
00:27:29 [INFO] [MultiVerse-Portals] - Version 2.0-b254 Enabled - By Rigby and f
ernferret
00:27:30 [INFO] [MultiVerse-Portals] 6 - Portals(s) loaded
00:27:30 [INFO] [BukkitInventoryTools] PermissionsBukkit/Permissions3/Permission
sEx system not detected, defaulting to permissions in plugin.yml
00:27:30 [INFO] [BukkitInventoryTools] Spout is detected.
00:27:30 [INFO] [BukkitInventoryTools] SQLite Initializing
00:27:30 [INFO] [BukkitInventoryTools] SpoutBackpack is detected.
00:27:30 [INFO] [BukkitInventoryTools] BukkitInventoryTools version 2.3.4 is ena
bled!
00:27:30 [INFO] [mcMMO] Using Bukkit Permissions.
00:27:30 [INFO] Loading Config File...
00:27:32 [INFO] mcMMO version 1.1.15 is enabled!
00:27:32 [INFO] mcMMO is keeping usage stats an. To opt-out for whatever bizarre
reason, check plugins/stats.
00:27:32 [INFO] [AdminCmd] Plugin Enabled. (version 5.8.1 (BUILD 2011-09-01 13/5
4/17))
00:27:32 [INFO] [AdminCmd] 80 extra help entries loaded from files: AdminCmd(80)
00:27:32 [INFO] [AdminCmd] Disabled Alias(es) for bal_repair : repair,
00:27:32 [INFO] [AdminCmd] Disabled Alias(es) for bal_sp : sp,
00:27:33 [INFO] [AdminCmd] 0 blacklisted items loaded.
00:27:33 [INFO] [AdminCmd] 1655 alias loaded.
00:27:33 [INFO] [AdminCmd] 0 Banned players loaded.
00:27:33 [INFO] [AdminCmd] 0 Muted players loaded.
00:27:33 [INFO] [AdminCmd] 2 kits loaded.
00:27:33 [INFO] [SpoutBackpack] Payment method was disabled. No longer accepting
payments.
00:27:33 [INFO] [SpoutBackpack] Version 2.3.4 is now enabled.
00:27:33 [INFO] [SpoutBackpack] Inventories loaded.
00:27:33 [INFO] Server permissions file permissions.yml is empty, ignoring it
00:27:33 [INFO] Done (5.280s)! For help, type "help" or "?"
00:27:38 [INFO] CButD: Plugin-Update-Check started
00:27:40 [INFO] No source for InventorySave.jar ask the Author to add it to dev.
bukkit.org
00:27:41 [INFO] No source for Safe Creeper.jar ask the Author to add it to dev.b
ukkit.org
00:27:41 [INFO] CityWorld.jar changed, try to download the new
00:27:41 [INFO] CityWorld.jar updated, copy D:\Games\Adventure\MineCraft.craftb
ukkit\bukkit_update\plugins\CityWorld.jar to your Plugin-Folder
00:27:42 [INFO] Multiverse-Core.jar changed, try to download the new
00:27:43 [INFO] Multiverse-Core.jar updated, copy D:\Games\Adventure\MineCraft.
craftbukkit\bukkit_update\plugins\Multiverse-Core.jar to your Plugin-Folder
00:27:43 [INFO] CButD: 2 plugins were updated, please copy them into your plugin
s folder.
00:27:43 [INFO] CButD: Plugin-Update-Check finished
I think this is due to the fact that the file format for a plan is pretty darn huge and inefficient.
My plan is this:
- Schedule saving until 5 seconds after no work has been done on the plan to minimize file opening/closing
- Split PlanArea's into smaller PlanChunks
- Implement lightweight, single number based arrays for the plan & original blocks instead of the current somewhat heavier object based version.
- Define a proper file format for the raw data chunks. This will probably use 3 bytes per block (1 for material, another for data and one more for good measure) instead of the current super overloaded strings.
- Saving will involve using RandomAccessFiles, so it will only overwrite the changed blocks and not the whole file when things change.
This should drastically reduce the memory footprint and increase performance a billion-fold.
Oh, and I will write a converter that will take the old plan format and convert into the new version so that no one loses their plans.
In the future I might look into using some kind of run-based compression, but it might affect save performance too much for it to be practical.