Issue Description:
After upgrade spigot server from 1.17.1 to 1.18.1.
Dynmap has rendring bug on the border of new chunk and old chunk.
New chunk or old chunk will not be rendering according to rendering starting point.
If you start rendring from new chunk, all new chunk will be redered. But old chunk will remain blank.
If you start rendring from old chunk, all old chunk will be rendered. But new chunk will remain blank.
Let's test it on the border:
Assume x=-2863,z=-2965 already visted by player which would update to new chunks, and outside is old chunks which already rended.
/dynmap radiusrender world -2863 -2965 1000
Then the new chunk will be rendering with a large black border between new chunk an old chunk.

• Dynmap Version: dynmap v3.3-SNAPSHOT-578
• Server Version: This server is running CraftBukkit version 3360-Spigot-f65de01-c2e0bbd (MC: 1.18.1) (Implementing API version 1.18.1-R0.1-SNAPSHOT)
• Pastebin of Configuration.txt: https://pastebin.com/eGfLRZBh
• Server Host (if applicable): McProHosting, Shockbyte, Selfhosted, etc.
• Pastebin of crashlogs or other relevant logs: N/A
• Other Relevant Data/Screenshots:
  
  
• Steps to Replicate:
• 1. Host a spigot 1.17.1 and explore large area
• 2. Upgrade to spigot 1.18.1 with old 1.17.1 map, and explore small area
• 3. Using new Dynmap plugin to fullrender or radiusrender.
• 4. If you start rendring on old chunks, all old chunks will be rended but remain blank on new chunks.
• 5. If you start rendring on new chunks, all new chunks will be rended but remain blank on old chunks.
• 6. If you start radius rendering on blank chunks... it may start rendering but with a large border between new chunk and old chunk.

Current Situation: (After fullrender on world)
Flat has only rendering new chunks.
Surface rendering new chunk and old chunks but has a border between new and old chunks.

Flat:

Surface:

Recommended process for upgrading from 1.17.1 to 1.18 (per Spigot) is:
<
World Upgrading
Although upgrades from between any two version pairs are technically possible, it is recommended that all upgrades are conducted from 1.17.x. This means that you should first upgrade your world completely on 1.17.x using the --forceUpgrade server argument, and then do the same on 1.18.

Have you followed this process?

ok... I will try it later!!
$ java -Xms8G -Xmx8G -jar spigot-1.18.1.jar --forceUpgrade nogui

forceUpgrade command may force update all chunk to 1.18.1
so... It's definitely no problem since there is no old chunk.

Thank you!!

So, spent some time looking at the issue folks upgrading from 1.17.1 to 1.18.x are seeing, and I'm getting a bad feeling that we are seeing how 'the sausage is made', as far as the chunk upgrade/migration process implemented by the vanilla server code. I'm not sure, but it is sure looking like their definition of migrating might be 'if the chunk has never been touched, then discard it and just allow worldgen to regenerate it when needed'. Need to prove it, which will involve additional analysis, but I'm getting the impression that that is what is happening - basicallly, if you do --forceUpgrade on a 1.17.1 world you haven't done anything with (I just fired up a clean 1.17.1, went creative, flew around a bit, rendered it, but no block interactions), when I bring up 1.18.1 with --forceUpgrade (or without it, having done it once), I'm only seeing the initial chunks around spawn (which, of course get loaded and 'finished' by being loaded). When I read the chunk data further out, I am not getting anything (or I'm getting proto-chunks that are not in the right state to be renderable). I really need to look at the world state with an independent tool - probable one of the NBT based world editor apps - to confirm the world data state after the --forceUpgrade. But, the border-of-black thing on a world that is old and has NOT had --forceUpgrade run would be consistent with this theory - any chunk that is considered 'pristine' and can (vanilla-wise) be regenerated versus migrated would start as a proto-chunk on the border so '1D - wise', the world would look like:

loaded 1.18.1 chunk -> (one or more) 1.18.1 protochunks (possibly replacing a discarded 1.17) -> old 1.17.1 chunks

I am not SURE this is the case (yet), but I wanted you guys to know that, if this IS the case, there will not be much I can do about it directly... I am considering (and have considered in the past) writing a mod for the sake of 'really' updating all chunks that are downlevel on the server by scanning the world data and basically forcing the chunks to get loaded and unloaded, but even that will not help if this --forceUpgrade procedure works as I am concerned it may.

we have the same problem ... we did forceUpgrade at the first start of 1.18 (not counting the testserver)

OK - so, it's weirder than I thought, but I'm more optimistic that I can sleaze something around it:

• When 1.18.x migrates a chunk (whether forced or not), it sets the chunk to the 'Empty' state: this is bizarre, as this is the first state that a newly created chunk would be in, even though the rest of the NBT data for the existing chunk is still there and has been translated.
• Player activity then drives the chunk to wind up in the normal 'full' state - whether or not that actually changes anything
  So, basically, I can get a map mostly working by rendering chunks that are cooked enough (normally 'light' state or later - 'full' being optimal), OR if they are in the 'empty' state but have data....
  The one gotcha here is that the 'empty' chunks may or may not have lighting data, which can result in rendering of chunks that look dark - working out some logic around that now, but I hope to have something to try shortly....

It take 2 hours for forceUpgrade to 1.18.1 from 1.17.1 with map size 16GB.
I will start a fullrender later then see the rendering result.
Thanks!!

Oh No...
After forceUpgrade from 1.17.1 to 1.18.1. (map keeps same size of 16GB)
And then fullrender entire world... Only 1.18.1 new chunk are rendering
Old Chunk is blank now!!

My Procedure of upgrade:

1. host spigot 1.17.1 and explore large area by player.
2. upgrade to spigot 1.18.1 without forceUpgrade and explore some area by player.
3. using Chunky plugin to force Generate circle radius 5000 blocks.
4. using spigot 1.18.1 with --forceUpgrade command and wait 2 hours for upgrade map.
5. fullrender world
6. now... only chunk within radius 5000 blocks and player visted chunk is rendered.

before --forceUpgrade:

After --forceUpgrade:

try snapshot #584 or later

ok, I am running fullrender with 3.3-SNAPSHOT-584 right now.
Hope this build can fix the problem.

Here is my site: http://cargon.net:8123/

Snapshot-584 fix the problem between old chunk and new chunk border.
But has a bug on the border between old chunk and blank area. (not generated area)
It seems the light issue you mention above.

Old chunk that player visited before:

Zoom In, Seems the light issue:

I'm having trouble reproducing what you are seeing. I specifically started a fresh 1.17.1 server, explored a bit, shut it down, installed 1.18.1 without using forceUpgrade, and did a clean render without seeing what you are seeing - both on borders from 1.18.1 converted and blank, old 1.17.1 not yet converted and blank, and on 1.17-1.18 borders.
The lighting you are seeing doesn't really make sense - there is lighting data there, but not consistent with that normally generated by vanilla code (the 'fade' there is from lighting data - but not anything my new code OR vanilla would generate). Any chance you've used a tool like worldborder or chunky or the like to drive lighting on chunks previously?

OK - I think I've got a lead on this - needed to go back to 1.16.x level chunks to see it though. I'll see what I can do, but the whole '--forceUpgrade' thing recommended by the Spigot guys might be a thought.

OK - give newest SNAPSHOT (585 or later) a go when you have a chance.

I just use Chunky to generate the circle inside 5000 radius.
The problem is outside 5000 radius.
There is no lighting fade out problem on the border of 5000 radius. (to old chunk and to blank area)

Does --fullUpgrade damage the lighting on old chunk?
I have upgrade dynamp to 3.3-SNAPSHOT-587
And try to radiusrender on the area that lighting fade out.
/dynmap radiusrender world -5000 3000 1000

Still have the same problem:
http://cargon.net:8123/?worldname=world&mapname=flat&zoom=4&x=-5000&y=64&z=3000

I am using fullrender right now.
Hope fullrender can solve the problem.

@mikeprimm Thanks for your great work!!

Well when I updated my spigot-server from 17.LAST to 1.18.1 yesterday, I had found the same black chunks very soon (using dynmap 3.3 Beta 2 jar). Then I started a fullrender in hope that would fix it but it became even wrose. Next I googled and found this. After reading I did this 'forceUpgrade' and well the server also did that. As next step I started a new fullrender but now it only rendered 5100 tiles after 5 minutes and proclaimed to be completed but map is now almost completly black and 5100 tiles is like a joke for the map size - any ideas what I could do now?

Snipped from the rendering process

Current Map Status:

An picture from an older Status

Start by running latest SNAPSHOT for 3.3 from here - https://dynmap.us/builds/dynmap/ - without reporting the specific version of dynmap you are running, there is very little to be further discussed. I have created multiple servers with extensive worlds around 1.17.1 and 1.16.5, upgraded to 1.18.1 (all around current Spigot builds), added 1.18.1 player activity to pull the chunks forward to 1.18, and tried with and without having run the 1.18.1 server with --forceUpgrade and not, and still cannot reproduce what you guys are reporting (and ditto for other folks I've worked with on the upgrade..).

The latest snapshot includes two new diagnostic 'shaders' - 'chunkversion' and 'chunkstatus' (see https://github.com/webbukkit/dynmap/blob/v3.0/DynmapCore/src/main/resources/shaders.txt#L198) that can be used to create color coded maps to show what version your chunks are, and what chunk status they have (status is relative to the worldgen lifecycle). Maps can be added for these using a very low res flat perspective:

dmap mapadd :version perspective:iso_S_90_vlowres shader:chunkversion lighting:shadows

dmap mapadd :status perspective:iso_S_90_vlowres shader:chunkstatus lighting:shadows

Then, a radiusrender of one of the affected areas might offer some insight - (if the areas show black in the status and version maps, switch the lighting setting to 'default' (for no lighting) or 'brightnight' for minimum lighting.

Status code color map is here - https://github.com/webbukkit/dynmap/wiki/Chunk-Status-Colors
Version color map is here - https://github.com/webbukkit/dynmap/wiki/Chunk-Version-Colors

Just updated to the 3.3 Beta 3 from 18.12; now it looks a bit better; at least it's not almost only black

I'm having the same issue, except I'm migrating from 1.12.2 Paper to 1.17.1 PurPur.
Ive been told on many occasions never to use --fullUpgrade as this can break a lot of things including chests etc, the current version of PurPur is extremely light on converting chunks from old to new when players visit them.

I have not expanded my world, all my chunks have been explored for years and years, my maps also been fully rendered for years until I moved from paper 1.12.2 to PurPur 1.17.1, i'm using Dynmap 3.3-beta-3-591

I changed the texture-pack, just so that I could visualize what was actually being re-rendered for debugging.

My live map https://map.piratemc.com/

None of the black void areas on the dynmap were there before, I used to have a perfectly rendered map.

Screenshot example of an admin flying around regenerating chunks for new terrain from 1.12.2 to 1.17.1.

The new resource pack is the darker one, so you can tell what has been newly generated from whats been flown over/triggered by staff.

Chunks will dynamically change as players explore them, maybe you need to hook into this mechanic in Paper/PurPur so that you can render those chunks as they are being explored and converted.

Once Ive migrated my server fully from 1.12.2 I will be testing on 1.18, at which point I will expand my world and run chunky on the world to see if I can force all chunks to be loaded and see if that makes a difference.

Just updated to the 3.3 Beta 3 from 18.12; now it looks a bit better; at least it's not almost only black

Well after a while in full render I can say, that the surface seems to need calculate 100% anew; server is still on it but what I could see so far is good;
The black lines; well sometimes a few of them spawns and disappere when I zoom in and out; guess that is nothing important; but some of the black lines keep staying. After I zoomed in I could find out that it seemes to be the same light bug that CardLin descriped before - but this can easily fixed by giving the chunck a light update - also for example just put a tourch on it and remove it and dynmap will fix it; as I have a lot of them; especialy close to the map's current borders - I will only remove the most anyoing one close to the centre and wait for the amulett editor to update to 1.18.1 for a large relighting order.

And here this pic that proofs that it works somehow

Give latest snapshot (600) a spin, if you have a chance (https://dynmap.us/builds/dynmap)

Got sent here by Kosmolot.

Running paperMC git-paper-68 and dynmap 3.3-beta-3-591

Artifacts happen on new 1.18 terrain, it has nothing to do with upgrading from 1.17

Not completely generated chunks render causing weird looks. Those chunks for instance are missing trees, frozen water, snow, grass etc. The biome coloring is also off - i think it's best to just ignore rendering of not completely generated chunks. When visiting the area, the chunks completely generate and dynmap automatically re-renders the tiles - fixing it. Though it would still be better to show black empty space than this weird border. I got scared that my world is getting corrupted.

Hopefully nobody minds if I chime in, experienced the same issue on my 1.18.1 Spigot server ever since running the new Dynmap beta.

Installing the latest snapshot (V3.3 Build 606) it is possible to somewhat remedy.
Teleporting to the chunk where the shadow appears & either running a /dynmap radiusrender XXX or simply staying put for a bit triggers the removal of the shadows.

Eventually this all clears up :

Lines are left in between the tiles, can't yet confirm if this was the case before; take this at face value.

I'll spin up a backup from 1.17 later today to confirm, I'm currently downloading said backup..

Important note I have never gotten dark spots and/or missing chunks, only shadows!
None of these shadows appear in-game, from what I recall.

New set of updates in SNAPSHOT 611 - this is all about dealing with the fact that both vanilla version upgrades and many of these other tools (worldborder and others) screw up lighting data on chunks that were previously lit, and rely on player activity to drive the chunks to become properly lit again.... basically, I've needed to write a basic lighting engine to generate sky lighting data for any chunks that are not back to 'light' status or better. This will wind up applying to chunks that were pulled forward by --forgeUpgrade and have not yet been visited again, for example, besides the usual border/edge chunks for newly generated areas. The migration process for chunks from 1.17 (or earlier) - whether forced or not - actually causes perfectly good lighting data to get trashed, while also causing invalid (gradient shaded) lighting data to be padded in the chunks before the chunks get properly lit... complete pain in the butt, honestly.

Snapshot 611 fixes all the issues I was previously having, so first & foremost; thank you for your hard work!

There are still small 4x4 block light glitches every now & then, however they seemingly fixed themselves after leaving the fullrender running overnight (Isometric isn't done but didn't display these 4x4 glitches, top down is now perfect like it was before!)

Is there any info or help I could provide? If so, I've got some spare time on my hands during the holiday's season anyways.

• Sven

I have a quick question on this topic because I also have the same problem with the Dynmap. As I have learned, there is now a solution that is included in the latest snapshot. I might be acting a bit stupid now, but where exactly can I find the Snapshot 611?
Thanks for all helpful answers.
-Hannes

Here you go bud
You can go through their build server here and find what you need manually in the future.

Might be worth saying, they're experimental pre-releases for a reason; use them at your own risk :)

Development builds are Dynmap-3.3-SNAPSHOT-*.jar files found at https://dynmap.us/builds/dynmap - these are RAW development builds: I don't tend to push code that hasn't been tested at least a little, BUT do understand that snapshots are minimally tested and can, from time to time, be broken.

@Sfekke Thanks for the test! Yep, when my basic lighting engine (which was written a couple of days ago to deal with how broken the lighting behavior in the chunk migration process is for existing chunks... see previous posts on this) is sky lighting the chunk, I do not currently do all the 'side scatter' of lighting that the full engine does (basically, sky light 'falls' vertically top to bottom, but then there is a second pass that I don't do to 'scatter' it to the sides, such that any non-solid block will have light of no less than 1 level below any of its lit neighbors). I'm probably going to do a simple version of this (the full version has light spilling from chunk to chunk, and ultimately cannot work well outside of full world gen or being limited to being done in chunks which are loaded along with all of their adjacent neighbors being loaded), but I need to see if I can do it without it killing performance more than me needing to generate light for broken light chunks is already an impact.

Net-net: my current simple lighting, which is used to fix all this broken lighting on chunks that are incompletely migrated, will cause solid blocks overhanging empty blocks to get very dark shadows, because the horizontal blocks that are lit do not currently share light to the sides (so caves and overhangs are overly dark)

I installed the snapshot and the black shadows are gone. Thank you (: What I also noticed is that the problem with the black shadows can also be found on the nether map. There are dark shadows at the edges of the map. I just wanted to say that briefly. -Hannes

The rendering problem on the "world" is solved. (version 3.3-SNAPSHOT-614)
But the "world_nether" have the same problem as @Hasenspringer says.

The rendering problem on the "world" is solved. (version 3.3-SNAPSHOT-614) But the "world_nether" have the same problem as @Hasenspringer says.

you need to rerender world_nether and world_the_end too

I did do a complete (flat) rerender of all my dimensions on my Fabric server, and it still isn't fixed completely.
dynmap version: 3.3-SNAPSHOT-618.
The shadow problem seems to be fixed, except for The Nether (Overworld and The End seem fine).
It is still rendering unpopulated chunks (as I did mention in #3574)
The chunk is clearly unpopulated because the trees are missing

The chunk is unpopulated because the ice and snow is missing

The Nether still has problems with lighting