The database had a few million entries and was over a gigabyte, and sometimes the server would get totally frozen up waiting on the PlayerLogger to write something to disk, and it was doing so at 1.5 MB/s with 80% I/O utilization. Only fix was to kill the server and start it up again, at which point I could enter the age without issue.

Sounds like a database issue (Just too much stuff). I will look into fixing the issues with sql so you can use a dedicated sql server with player logger.

Also, I can look allowing players to disable the logger in specific dimensions as a work around.

Sorry for the inconvenience. This is simply a limitation of sql lite when dealing with big data.

Well if I should use MySQL, what's with the "DO NOT USE MYSQL UNLESS YOU REALLY NEED TO!" comment? And why is so much data being generated anyways?

The reason MySQL is discouraged is because it is buggy and not much time has been spent to optimize FE for it. However, this is a situation where mysql might be beneficial. However, I am planning to take a deep dive into FE's database systems next week and see if there are places where databases like hbase or kafka+avro would be more beneficial.

And I will look into why so much stuff is generated when I take a look at fixing this stuff.

After some further checking, it turns out that only about 6,000 out of 600,000 entries are explosions of actual blocks. The rest are air blocks being exploded. The whitelist, while probably still useful, may not be strictly necessary to fix this issue.

@spacebuilder2020 that's a little wrong.
Playerlogger works perfectly with MySql - it just changes the connection (the problems for SQL were with permissions).

In this case it's a simple matter of performance.
MySql - or in general a dedicated SQL server is SLOW. REALLY SLOW.

Compared to that the h2 database running in embedded mode (inside the MC server) which does not need to be able to handle multiple connections and a lot of other stuff is way faster.

The problem you experienced is probably some world generation problem.
Maybe Mystcraft generates some structures into the world after it has been initially generated, causing playerlogger to try to log all that, basically stalling the whole server.

We will have to investigate what is actually causing those large amounts of playerlogger data.
It would be great if you could create a new empty DB (delete the old one) and then repeat what caused the problem.
If the server stalls again, monitor the db file and see if it gets larger (though the OS might not be able to show that directly) and kill the server again before it gets excessively large.

Then you could send us a copy of your DB so we can investigate the issue.

@spacebuilder2020 another thing we could do would be adding playerlogger visualization to the FE-client mod which would allow @Techjar to actually see what was logged after restarting the server (without the need to send us his DB).

That is why I am going to take a look at hbase and kafka.

And when I look at player logger, I plan to make it far more functional.

Yeah, I'll do some testing on a local server later to try and find out what's filling the playerlogger.

Btw, you should move your player logger to a dedicated hard drive. Preferably an ssd.

This is on a rented dedicated box, so that's not really an option. But until messing with Mystcraft it hadn't been an issue.

In general it's not really necessary to put playerlogger on another drive - only once you have things like many quarries running on a server etc. or a very slow HDD.

So turns out it is in fact exploding, literally. Hehe, jokes aside, it actually seems to be caused by the ridiculous amount of explosions in an unstable Mystcraft age. So the solution I think is to add a whitelist (not a blacklist, that won't help here as you can't possibly blacklist every new dimension), and have it be empty default, where empty is interpreted as log all dimensions, so that it won't change existing behavior.

Also, a major contributor to the massive amount of data (as in basically all of it) seems to be air blocks being blown up and logged. I've create a separate issue for that (#2137).