[Request] - Ability to specify where the `database.db` file is stored.

Question

[Request] - Ability to specify where the `database.db` file is stored.

MrRazamataz opened this issue 3 years ago · 21 comments

MrRazamataz commented 3 years ago

For example, on a different drive, as the database files can get quite chonky.

database_file_location:
      `/a/new/location/database.db

Like with a pterodactyl mount.
Thank you.

Intelli · Answer 1 · 2021-10-29T21:55:21.000Z

CoreProtect supports custom plugin directories. You'd have to move the entire plugins directory, but it's the recommended method if you want data inside the plugins folder on a different drive.

Start-up Parameter:

--plugins
Alias: -P
Default: plugins
This parameter allows you to manually define the plugins directory to use for the server.
Please note, pay attention to the uppercase for the alias.

The other option would be to use MySQL, if you wanted only CoreProtect data to be on another drive.

MrRazamataz · Answer 2 · 2021-10-30T01:24:09.000Z

Moving the entire plugins directory for my server isn't something that would work well for me, because of plugins that depend on file structure. This is why I requested a custom database location. I also could use a MySQL database, but this would mean I would either have to reset my data, or attempt to convert the data from SQLite to MySQL, which doesn't sound nice or like it would work.

MrRazamataz · Answer 3 · 2021-11-03T01:05:17.000Z

Moreover, this would work better with mounts such as pterodactyl mounts, where you can't view the files through the file manager or SFTP (unless logging into the machine itself, outside of ptero) but the server can access the files. This would make mounts possible with CoreProtect.

molor · Answer 4 · 2021-11-07T09:39:11.000Z

I also have a big database file, and ability to move only it (as in the case with dynmap, where I have already moved the database to the big HDD, leaving the server on the small SSD) would be a good thing..

Any news?

Intelli · Answer 5 · 2021-11-07T22:27:20.000Z

Any news?

The present available options are to either move your plugins folder to the HDD, or to set up a MySQL database on the HDD. Both are viable options that are already available.

A symbolic link would also work if you just want to just move the database.db file. Note though that running a purge would overwrite the symlink with a full copy of the database.

MrRazamataz · Answer 6 · 2021-11-11T23:35:39.000Z

Any news?

The present available options are to either move your plugins folder to the HDD, or to set up a MySQL database on the HDD. Both are viable options that are already available.

A symbolic link would also work if you just want to just move the database.db file. Note though that running a purge would overwrite the symlink with a full copy of the database.

I tried creating a symbolic link, got it all setup, and CoreProtect failed to enable.

[00:27:09] [Server thread/INFO]: [CoreProtect] Enabling CoreProtect v20.1
[00:27:09] [Server thread/WARN]: org.sqlite.SQLiteException: [SQLITE_CANTOPEN]  Unable to open the database file (unable to open database file)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.DB.newSQLException(DB.java:1012)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.DB.newSQLException(DB.java:1024)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.DB.throwex(DB.java:989)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.NativeDB._open_utf8(Native Method)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.NativeDB._open(NativeDB.java:78)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.DB.open(DB.java:195)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:243)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.SQLiteConnection.<init>(SQLiteConnection.java:61)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.jdbc3.JDBC3Connection.<init>(JDBC3Connection.java:28)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.jdbc4.JDBC4Connection.<init>(JDBC4Connection.java:21)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.JDBC.createConnection(JDBC.java:115)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.JDBC.connect(JDBC.java:90)
[00:27:09] [Server thread/WARN]: 	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
[00:27:09] [Server thread/WARN]: 	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:251)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.database.Database.getConnection(Database.java:174)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.database.Database.getConnection(Database.java:130)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.database.Database.createDatabaseTables(Database.java:373)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.config.ConfigHandler.loadDatabase(ConfigHandler.java:205)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.config.ConfigHandler.performInitialization(ConfigHandler.java:367)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.CoreProtect.onEnable(CoreProtect.java:71)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:264)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:370)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:500)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.craftbukkit.v1_17_R1.CraftServer.enablePlugin(CraftServer.java:561)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.craftbukkit.v1_17_R1.CraftServer.enablePlugins(CraftServer.java:475)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.loadWorld(MinecraftServer.java:732)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.dedicated.DedicatedServer.init(DedicatedServer.java:328)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.x(MinecraftServer.java:1221)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.lambda$spin$0(MinecraftServer.java:321)
[00:27:09] [Server thread/WARN]: 	at java.base/java.lang.Thread.run(Thread.java:831)
[00:27:09] [Server thread/WARN]: java.lang.NullPointerException: Cannot invoke "java.sql.Connection.createStatement()" because "connection" is null
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.database.Database.createDatabaseTables(Database.java:374)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.config.ConfigHandler.loadDatabase(ConfigHandler.java:205)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.config.ConfigHandler.performInitialization(ConfigHandler.java:367)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.CoreProtect.onEnable(CoreProtect.java:71)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:264)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:370)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:500)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.craftbukkit.v1_17_R1.CraftServer.enablePlugin(CraftServer.java:561)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.craftbukkit.v1_17_R1.CraftServer.enablePlugins(CraftServer.java:475)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.loadWorld(MinecraftServer.java:732)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.dedicated.DedicatedServer.init(DedicatedServer.java:328)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.x(MinecraftServer.java:1221)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.lambda$spin$0(MinecraftServer.java:321)
[00:27:09] [Server thread/WARN]: 	at java.base/java.lang.Thread.run(Thread.java:831)
[00:27:09] [Server thread/WARN]: org.sqlite.SQLiteException: [SQLITE_CANTOPEN]  Unable to open the database file (unable to open database file)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.DB.newSQLException(DB.java:1012)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.DB.newSQLException(DB.java:1024)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.DB.throwex(DB.java:989)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.NativeDB._open_utf8(Native Method)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.NativeDB._open(NativeDB.java:78)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.core.DB.open(DB.java:195)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:243)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.SQLiteConnection.<init>(SQLiteConnection.java:61)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.jdbc3.JDBC3Connection.<init>(JDBC3Connection.java:28)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.jdbc4.JDBC4Connection.<init>(JDBC4Connection.java:21)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.JDBC.createConnection(JDBC.java:115)
[00:27:09] [Server thread/WARN]: 	at org.sqlite.JDBC.connect(JDBC.java:90)
[00:27:09] [Server thread/WARN]: 	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677)
[00:27:09] [Server thread/WARN]: 	at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:251)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.database.Database.getConnection(Database.java:174)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.database.Database.getConnection(Database.java:130)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.config.ConfigHandler.performInitialization(ConfigHandler.java:369)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.CoreProtect.onEnable(CoreProtect.java:71)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:264)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:370)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:500)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.craftbukkit.v1_17_R1.CraftServer.enablePlugin(CraftServer.java:561)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.craftbukkit.v1_17_R1.CraftServer.enablePlugins(CraftServer.java:475)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.loadWorld(MinecraftServer.java:732)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.dedicated.DedicatedServer.init(DedicatedServer.java:328)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.x(MinecraftServer.java:1221)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.lambda$spin$0(MinecraftServer.java:321)
[00:27:09] [Server thread/WARN]: 	at java.base/java.lang.Thread.run(Thread.java:831)
[00:27:09] [Server thread/WARN]: java.lang.NullPointerException: Cannot invoke "java.sql.Connection.createStatement()" because "connection" is null
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.config.ConfigHandler.performInitialization(ConfigHandler.java:370)
[00:27:09] [Server thread/WARN]: 	at CoreProtect-20.1.jar//net.coreprotect.CoreProtect.onEnable(CoreProtect.java:71)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.java.JavaPlugin.setEnabled(JavaPlugin.java:264)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.java.JavaPluginLoader.enablePlugin(JavaPluginLoader.java:370)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.plugin.SimplePluginManager.enablePlugin(SimplePluginManager.java:500)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.craftbukkit.v1_17_R1.CraftServer.enablePlugin(CraftServer.java:561)
[00:27:09] [Server thread/WARN]: 	at org.bukkit.craftbukkit.v1_17_R1.CraftServer.enablePlugins(CraftServer.java:475)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.loadWorld(MinecraftServer.java:732)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.dedicated.DedicatedServer.init(DedicatedServer.java:328)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.x(MinecraftServer.java:1221)
[00:27:09] [Server thread/WARN]: 	at net.minecraft.server.MinecraftServer.lambda$spin$0(MinecraftServer.java:321)
[00:27:09] [Server thread/WARN]: 	at java.base/java.lang.Thread.run(Thread.java:831)
[00:27:09] [Server thread/INFO]: [CoreProtect] CoreProtect was unable to start.

MrRazamataz · Answer 7 · 2021-11-29T16:42:11.000Z

@Intelli are you sure a syslink works? see above error

Intelli · Answer 8 · 2021-11-29T23:36:42.000Z

Works fine on my system.

molor · Answer 9 · 2021-11-30T09:25:52.000Z

I'm ungrateful fool.

MrRazamataz · Answer 10 · 2021-12-01T21:44:44.000Z

I wouldnt say lazy, but I would like for my error message to be read, which shows it didnt work on my system.

MrRazamataz · Answer 11 · 2022-01-05T22:08:40.000Z

@silen72 Unfortunately, even after I redid the link, its still not working. This is the error here.

These are commands that I ran:

  731  mv /var/lib/pterodactyl/volumes/5d3c2312-0736-4461-a061-5e1ea79490ad/plugins/CoreProtect/database.db /media/nvme2/ptero/CoreProtect
  732  ln -s /media/nvme2/ptero/CoreProtect/database.db /var/lib/pterodactyl/volumes/5d3c2312-0736-4461-a061-5e1ea79490ad/plugins/CoreProtect/
  733  cut -d: -f1 /etc/passwd
I set as 777 via FileZilla, and it didnt work, so i tried all this:
  734  chmod pterodactyl:pterodactyl
  735  chmod pterodactyl
  736  chmod pterodactyl:pterodactyl /media/nvme2/ptero/CoreProtect/database.db
  737  chmod -R  pterodactyl:pterodactyl /media/nvme2/ptero/CoreProtect/database.db
  738  chmod --help
  739  chmod -R pterodactyl:pterodactyl /media/nvme2/ptero/CoreProtect/database.db
  740  chmod pterodactyl:pterodactyl /media/nvme2/ptero/CoreProtect/database.db
  741  chmod pterodactyl /media/nvme2/ptero/CoreProtect/database.db
  742  chmod root: pterodactyl /media/nvme2/ptero/CoreProtect/database.db
  743  chmod root:pterodactyl /media/nvme2/ptero/CoreProtect/database.db
  744  chown pterodactyl:pterodactyl /media/nvme2/ptero/CoreProtect/database.db
  745  chown pterodactyl:pterodactyl /var/lib/pterodactyl/volumes/5d3c2312-0736-4461-a061-5e1ea79490ad/plugins/CoreProtect/database.db

silen72 · Answer 12 · 2022-01-05T21:19:56.000Z

@MrRazamataz I´m far from being an expert but that seems like a pretty odd combination of commands to me. I've tried to reproduce the steps you provided but failed (expectedly):

ln - is the link command, a simple "ln" does nothing (except complain about missing file operand) -> fail
cp /sourcefolders/database.db /targetfolders - this copies the file 'database.db' to the target folder -> ok
ln -s /targetfolders/database.db /sourcefolders/ - this TRIES to link the copied database.db from the target folder back to the source folder, but can´t because the file 'database.db' is still there (ln: failed to create symlink '[...]': File exists) -> fail

from there it´s rather going downhill:

rm lrwxrwxrwx - that is "remove file 'lrwxrwxrwx'" which I bet failed: lrwxrwxrwx is a specification of access privileges and not a file -> fail
unlink lrwxrwxrwx - this is exactly the same as above: remove file 'lrwxrwxrwx' -> fail
ublink database.db - ublink is not a command (a typo?) -> fail
unlink database.db - remove file 'database.db' - this IS a valid command, but where did you issue that? in the target folder -> not sure
cd /sourcefolders/ - change to source folder -> ok
unlink database.db - remove database.db => this alone explains, why CoreProtect was "Unable to open the database file"

To sum up: if I where CoreProtect, I´d complain too - you have simply taken away my database ;-)

To do it right, I'd suggest you try something along the line of the following (obviously when your minecraft server is down):

move the database.db: mv /sourcefolders/database.db /targetfolders/
create the symlink: ln -s /targetfolders/database.db /sourcefolders/
double check that the user account the minecraft server is started with has write access to the database.db AND the containing folder (use something like chmod minecraft:minecraft on it, if 'minecraft' is the user account name)

See here for more.

Edit: I recommend practicing that in a virtual machine, at least that is what I do to understand, what each step does and to assure the desired result...

@molor With all that said, I think an apology to @Intelli is due for your pretty rude and inappropriate comment, don´t you think?

MrRazamataz · Answer 13 · 2022-01-05T21:37:35.000Z

@silen72 I just provided the entire history of that session on my machine (so yes that one was probably a typo and sorry for it being not so useful!)
For ln I was looking at the help, and for cp I was making a backup of the database and moving it to the other drive.
The other commands do seem incorrect looking back and at your comment :D. Im going to try the new commands that you have recommend and after reading that page.

silen72 · Answer 14 · 2022-01-05T17:11:21.000Z

@MrRazamataz, would you mind elaborating on HOW you set up the symlink? It might be that something went wrong there...

MrRazamataz · Answer 15 · 2022-01-05T19:26:53.000Z

@silen72 This is what I did (i googled it at the time)

  597  ln
  598  cp /var/lib/pterodactyl/volumes/5d3c2312-0736-4461-a061-5e1ea79490ad/plugins/CoreProtect/database.db /media/nvme2/ptero/CoreProtect
  599  ln -s /media/nvme2/ptero/CoreProtect/database.db /var/lib/pterodactyl/volumes/5d3c2312-0736-4461-a061-5e1ea79490ad/plugins/CoreProtect
  600  ls -l /var/lib/pterodactyl/volumes/5d3c2312-0736-4461-a061-5e1ea79490ad/plugins/CoreProtect/database.db
  601  ls -la /var/lib/pterodactyl/volumes/5d3c2312-0736-4461-a061-5e1ea79490ad/plugins/CoreProtect | grep "\->"
  602  rm lrwxrwxrwx
  603  unlink lrwxrwxrwx
  604  ublink database.db
  605  unlink database.db
  606  cd /var/lib/pterodactyl/volumes/5d3c2312-0736-4461-a061-5e1ea79490ad/plugins/CoreProtect
  607  unlink database.db

Its entirely possible I did something wrong but the file was created on the other end in the right place.

MrRazamataz · Answer 16 · 2021-12-11T17:42:21.000Z

@Intelli any update on this? It clearly doesnt work on my system

silen72 · Answer 17 · 2022-01-06T11:34:29.000Z

@MrRazamataz I just took a brief look at the Pterodactyl website, it says: "Pterodactyl is the open-source game server management panel built with PHP7, Nodejs, and Go. Designed with security in mind, Pterodactyl runs all game servers in isolated Docker containers [...]"

Docker isolates the inside of the container from the outside. As far as I understand, you are trying to access outside folders from inside a container. That won't happen and there is nothing CoreProtect (or minecraft or anyone else) could do about it. If you want to make outside folders or files accessible inside your container you need to make yourself familiar with the concepts of container virtualisation and configure your container accordingly. See here and there.

Maybe this is a pointer to motivate you to make the move towards a more robust and professional database backend ;-P

As for this request I strongly agree with @Intelli to close it: all options to store your data outside of the regular plugin folder are already available, there is no need to add another one:

custom plugin directory
move/symlink the database.db using pure os means (mv/ln on linux and Mac, mklink or Link Shell Extension on Windows)
MySQL / MariaDB
database.db as a volume with container virtualization

MrRazamataz · Answer 18 · 2022-01-09T00:46:01.000Z

@silen72 Sorry for the delayed response.
Yes the server is running in a docker container, but the location of the DB which is linked to is a mount in the pterodactyl server. I'm already using it for other plugins. I suggested this in the first place because then I could just specify the location of the mount in the config and it would work. Sorry I hadn't mentioned this before.

Mount:

The file location of the attempted DB is /media/nvme2/ptero/CoreProtect/database.db which is inside the mount. I know the mount works correctly because I have used it in other plugins.

The option of a MySQL database is valid, but I was hopig not to do that as it would involve messing with my (large number) of existing databases for other things, and converting a 44GB .db file sounds fun :P

I will take a look at the pure os means if its not going to work, but I would have thought that because it's in the mount location it would have.

Thank you

silen72 · Answer 19 · 2022-01-09T10:27:27.000Z

@MrRazamataz Take your time ;-)

What I see is that you mount a complete folder, where - among others - CoreProtect stores its stuff. What I was talking about is that you can (and in your case probably have to) mount a single file (database.db) as an overlay. That is the "container way" of having a symbolic link. Like this (mimicking your screenshot):
1 name1 /real/source/folder /virtual/folder/in/container
2 name2 /other/source/folder/database.db /virtual/folder/in/container/database.db

That way you tell docker to mount name1 and all files contained but database.db, which it shall mount from a completely different location. Inside the container you won't be able to tell the difference, database.db will look like any other file in that folder.

The other thing is that a container is a use-once-then-throw-away thing. So if you manage to create symlinks inside your running container and restart that container, your changes are gone. It's like booting a live linux from a CD, if you're familiar with that. That's why I supposed to make yourself familiar with the concepts of container virtualisation. I´m pretty convinced that once you grasp that, you will be able to achieve what you want.

MrRazamataz · Answer 20 · 2022-01-10T10:18:12.000Z

I will try and take a look at this if I get time.

I still think a config option for specifying the location would be easier and would work with mounts. :P

stale · Answer 21 · 2022-03-11T12:52:10.000Z

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Share to