I'm currently stumped on how to use the custom tree I made using the gui for the 1.14.4 version of this mod. Maybe have it explained in the GUI itself?

The easiest way currently is probably this mod by @Darkhax.
Then you just have to throw the files in the appropriate subdirectories. Read that mods description for more info.

And yes, an explanation is on the todo list.

I tried adding some custom trees and then moving it to the data folder using Open Loader, but I couldn't get it to function.

An explanation is now included in the GUI, along with some error handling.
Update to the latest version 2.1.2 to get those features.