Tile generation halts after several hundred tiles.

• Dynmap Version: dynmap v3.1-beta-2-389
• Server Version: Spigot 1.16.1

[09:37:10] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 100 tiles rendered (92.35 msec/tile, 55.73 msec per render)^[[m
[09:37:18] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 200 tiles rendered (85.99 msec/tile, 53.50 msec per render)^[[m
[09:37:25] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 300 tiles rendered (78.82 msec/tile, 47.06 msec per render)^[[m
[09:37:33] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 400 tiles rendered (75.78 msec/tile, 43.85 msec per render)^[[m
[09:37:40] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 500 tiles rendered (74.18 msec/tile, 42.84 msec per render)^[[m
[09:37:46] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 600 tiles rendered (72.44 msec/tile, 41.91 msec per render)^[[m
[09:37:53] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 700 tiles rendered (71.22 msec/tile, 41.35 msec per render)^[[m
[09:37:59] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 800 tiles rendered (69.70 msec/tile, 40.54 msec per render)^[[m
[09:38:06] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 900 tiles rendered (70.30 msec/tile, 40.21 msec per render)^[[m
[09:38:15] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 1000 tiles rendered (71.43 msec/tile, 40.81 msec per render)^[[m
[09:38:22] [Dynmap Render Thread/INFO]: Full render of map 'flat' of 'world' in progress - 1100 tiles rendered (71.51 msec/tile, 40.60 msec per render)^[[m
[09:38:30] [Dynmap Render Thread/ERROR]: [dynmap] Exception while fetching chunks:
java.lang.IllegalArgumentException: Invalid length given for storage, got: 90 but expected: 64
        at net.minecraft.server.v1_16_R1.DataBitsPacked.<init>(SourceFile:39) ~[mur_creative.jar:git-Spigot-0509002-7c03d25]
        at org.dynmap.bukkit.helper.v116.MapChunkCache115$NBTSnapshot.<init>(MapChunkCache115.java:220) ~[?:?]
        at org.dynmap.bukkit.helper.v116.MapChunkCache115.loadChunks(MapChunkCache115.java:449) ~[?:?]
        at org.dynmap.bukkit.DynmapPlugin$BukkitServer$6.call(DynmapPlugin.java:490) ~[?:?]
        at org.dynmap.bukkit.DynmapPlugin$BukkitServer$6.call(DynmapPlugin.java:477) ~[?:?]
        at org.bukkit.craftbukkit.v1_16_R1.scheduler.CraftFuture.run(CraftFuture.java:88) ~[mur_creative.jar:git-Spigot-0509002-7c03d25]
        at org.bukkit.craftbukkit.v1_16_R1.scheduler.CraftScheduler.mainThreadHeartbeat(CraftScheduler.java:400) ~[mur_creative.jar:git-Spigot-0509002-7c03d25]
        at net.minecraft.server.v1_16_R1.MinecraftServer.b(MinecraftServer.java:1061) ~[mur_creative.jar:git-Spigot-0509002-7c03d25]
        at net.minecraft.server.v1_16_R1.DedicatedServer.b(DedicatedServer.java:354) ~[mur_creative.jar:git-Spigot-0509002-7c03d25]
        at net.minecraft.server.v1_16_R1.MinecraftServer.a(MinecraftServer.java:1009) ~[mur_creative.jar:git-Spigot-0509002-7c03d25]
        at net.minecraft.server.v1_16_R1.MinecraftServer.v(MinecraftServer.java:848) ~[mur_creative_.jar:git-Spigot-0509002-7c03d25]
        at net.minecraft.server.v1_16_R1.MinecraftServer.lambda$0(MinecraftServer.java:164) ~[mur_creative.jar:git-Spigot-0509002-7c03d25]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

Uncertain if this should be marked as a duplicate since I couldn't find similar issues related to Spigot 1.16.1.

We tested on the forge 1.16.1 version and also hit this issue:

Also there is #3068 which is probably the same issue.

I was trying to investigate why this issue was happening on the fabric port but it seems that it is 1.16 related after all.

EDIT: this comment is more or less obsolete. see the one below.

Did some more tests and found out something interesting.
The error is thrown by net.minecraft.util.BitArray. It calculates an expected data array length and if that doesn't match its actual length it will throw an exception.
Now to that expected data array length: the calculation algorithm changed between 1.15 and 1.16.
I compared the two algorithms here so you can see for yourself: https://scalafiddle.io/sf/iPq0wRH/2

val size = 4096
val interval = 64

def expected_length_1_15(bits: Int): Int = {
  val number = bits * size
  val i = number % interval
  (if (i == 0) number else number + interval - i) / interval
}

def expected_length_1_16(bits: Int): Int = {
  val number = (interval / bits).toChar
  (size + number - 1) / number
}

for (i <- 0 until 20)
  println(expected_length_1_15(i) + " " + expected_length_1_16(i))

The numbers I get out immediately explain why this issue occurs. I will try to fix this in the fabric port and if that works out I will submit a PR for the other platforms.

EDIT:
Okay I think now I know what's happening. The chunks which are affected by this are generated by a pre 1.16 version of minecraft. Once you actually visit these chunks they will get converted. Newly generated chunks should never be affected. So the workaround I implemented in the fabric port would actually be the right thing to do for the other versions. With this workaround dynmap will skip affected chunks instead of crashing. This will cause some areas to render like this until you visit them but this should be better than crashing I guess:

I just saw that you already figured out the stuff above in the spigot version. I will do the same for the fabric and forge version. That still doesn't explain this issue which happened on an already fixed spigot version. Maybe there are older "legacy encodings"? Maybe the workaround to skip affected chunks as proposed above should be the third fallback?

Mike has addressed this issue in a recent snapshot, you can download that here:
https://dynmap.us/builds/dynmap/Dynmap-3.1-SNAPSHOT-forge-1.16.1.jar

I'm going to close this and assume its fixed but please reply and I will reopen it if the issue persists.

@FedUpWith-Tech I think this issue is actually not fixed yet. While the two other "invalid length" issues were on forge this one is on spigot and in spigot the issue was already thought to be fixed by 17cbfb9 in dynmap v3.1-beta-1.

Also the numbers 90 and 64 don't line up with neither the old nor the new storage format. This is why I expressed my theory above that there might actually be a third storage format from a (much?) older mc version. Not sure about that one but I don't really have a better explanation where these numbers could come from.

What is the specific data on the reported scenario for reopening this? Unless I can be given a specific version and a specific background, there is nothing to work on here. The lengths aren't magic - they just are two different mappings of otherwise variable length data that don't necessarily match up - existing problems prior to the fix will include other combinations of numbers.

What isn't going to be supported is pre-1.13 chunks - that stuff has to be migrated by the MC/Forge server itself: the information needed to migrate those literally doesn't exist, and the mapping is absurdly complicated, as many vanilla blocks went from very different IDs and state mappings to newer ones (much less the logic of doing so for Forge mods providing custom blocks). Sorry - that isn't going to be doable by Dynmap, and the 1 in 100 folks who actually are doing that will need to use something to drive the migration of those chunks to the newer format.

Sorry for bringing this issue up again. Since I am not the original poster I don't actually know from which version the world was migrated. I originally just commented on this issue because I thought it had something to do with the other "invalid length" errors but that turned out to be false. That you are not supporting pre-1.13 chunks sounds totally reasonable to me.
I think there is no inherent fix to this issue here but maybe it would be a sensible workaround to catch the exception and letting dynmap skip affected chunks instead of letting the game crash. This is just a proposal which I also implemented for the fabric version in case there is someone trying to load very old world data. I am not going to bother you on this issue any more than I already have but just wanted to let you know this.

I'm actually tempted to write a plugin to do the operation I suggested - it's just not appropriate for Dynmap to do it, as I make a very major point of making sure Dynmap stays 'read only' in its role of processing world data. Frankly, don't have the time to deal with hundreds of server folks blaming Dynmap for corrupting their world data.... we already get blamed for every memory issue they encounter, because we've got a thread pool that includes labelled threads, so naturally every server crash is because of the fact that Dynmap's name is somewhere in the thread dump...

I certainly can handle the un-processable chunk as a null chunk with a warning post.

Fix was in 3.1-beta-3 - if anyone has a new report with that release (or, better, 3.1-beta-3a), open a new issue