Configuring Group Sync
GrnEyedDvl opened this issue · 10 comments
We have been using this for a couple of years now with no problem on older versions of Bukkit and PEX, the new version of CB will not sync correctly with groups. It doesn't even look like its pulling the usergroupid from the database table. Previously It was working with a custom userfield for MC names, now I cant even get it to work by pulling the forum username.
This is what it pulls, log level set to ALL:
[11:32:32] [User Authenticator #1/INFO]: [CommunityBridge] SELECT user.userid FROM user WHERE LOWER(username) = LOWER('GrnEyedDvl') ORDER BY userid DESC
[11:32:32] [User Authenticator #1/INFO]: [CommunityBridge] GrnEyedDvl linked to web application user ID #1.
[11:32:32] [User Authenticator #1/INFO]: UUID of player GrnEyedDvl is 59ee3255-89ac-4c69-a6ab-6190234c6f22
Now, if I have a group set in the registered-player-group: it will stick me in that group just fine and even includes the UUID. But it puts all players in that group. If I leave that blank it should try the group-sync options and take my forum user group and match it up with the groups defined here:
group-mapping:
'1' : 'default'
'2' : 'player'
'6' : 'admin'
But its not even pulling the usergroupid number as seen by the above query, so it has nothing to match it with.
If I setup the multi-tables options as such:
multi-tables: true
users-table:
table: users
user-name-field: username
user-id-field: userid
groups-id-field: displaygroupid
multi-table:
table: userfield
field-user-id-field: userid
field-value-field: field1
It still doesn't pull any information that allows it to link a forum group to a PEX group. I am running it off of displaygroupid because of how we have the forum setup, but it doesn't matter if I use the primary group either and change the values for the admin group to match that. Directly querying the database you can see I have the correct values set in the CB config file.
mysql> select username, userid, displaygroupid from user where userid = 1;
+------------+--------+----------------+
| username | userid | displaygroupid |
+------------+--------+----------------+
| GrnEyedDvl | 1 | 6 |
+------------+--------+----------------+
mysql> select userid, field1 from userfield where userid = 1;
+--------+------------+
| userid | field1 |
+--------+------------+
| 1 | GrnEyedDvl |
+--------+------------+
As a small aside, there is at least one bug I found that may or may not be related to this. Setting this to true:
simple-synchronization:
Set this to true for simple synchronization
enabled: true
Gives me this in the log
[11:32:19] [Server thread/INFO]: [CommunityBridge] Simple synchronization enabled : false
Am I missing a setting someplace?
You're in effect missing LOTS of settings.
The configuration snippets you've provided tells me you were using 1.x. The configuration file changed completely with 2.x. You'll want to move your old 1.x config.yml somewhere else (so you have it as a reference), and delete the plugins/CommunityBridge directory. Then when you start up MC, CommunityBridge will generate a blank config.yml. You can then set the appropriate settings in the new config.yml. It may be helpful to look at: http://dev.bukkit.org/bukkit-plugins/communitybridge-fm/pages/documentation/
A common mistake for folks coming from 2.x is altering the indentation of the config.yml. Yaml configurations are very sensitive to indentation, be careful to maintain the indentation as it is using spaces. Under Yaml, the following are not the same:
simple-synchronization:
# Set this to true for simple synchronization
enabled: truesimple-synchronization:
# Set this to true for simple synchronization
enabled: trueOk, against my better judgment I ran MC as root on Ubuntu 12.04. All plugins except PermissionsEx and CB are disabled.
Started with a fresh config file and edited it. The PEX file has two user groups in it, default and player. This almost works, running the server as root which I do NOT want to do.
It did not throw me in the player group like it should. Here is the log file, you can see it pulled the usergroupid, or at least tried to.
[21:36:01] [Server thread/INFO]: [PermissionsEx] Enabling PermissionsEx v1.22.1
[21:36:01] [Server thread/INFO]: [PermissionsEx] Initializing file backend
[21:36:01] [Server thread/INFO]: [PermissionsEx] Permissions file successfully reloaded
[21:36:01] [Server thread/INFO]: [CommunityBridge] Enabling CommunityBridge v2.3.7.454
[21:36:01] [Server thread/INFO]: [CommunityBridge] Detected locale: en
[21:36:01] [Server thread/WARN]: [CommunityBridge] The achievements.yml file is empty.
[21:36:01] [Server thread/INFO]: [CommunityBridge] Log level : all
[21:36:01] [Server thread/INFO]: [CommunityBridge] Plugin metrics enabled : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Use achievements : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Permissions system : PEX
[21:36:01] [Server thread/INFO]: [CommunityBridge] Economy enabled : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Autosync : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Autosync every : 20 minutes
[21:36:01] [Server thread/INFO]: [CommunityBridge] Synchronize during join event : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Synchronize during quit event : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Application url : http://www.example.org/
[21:36:01] [Server thread/INFO]: [CommunityBridge] Date Format : yyyy-MM-dd hh:mm:ss a
[21:36:01] [Server thread/INFO]: [CommunityBridge] Database hostname : localhost
[21:36:01] [Server thread/INFO]: [CommunityBridge] Database port : 3306
[21:36:01] [Server thread/INFO]: [CommunityBridge] Database name : forum_db
[21:36:01] [Server thread/INFO]: [CommunityBridge] Database username : mc_data
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking auto reminder : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking auto reminder every : 10 minutes
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking notify registered : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking notify unregistered : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking kick unregistered : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking unregistered group :
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking registered group :
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking notify player of group : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking reg former unregistered only : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking uses key-value pair : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking table name : user
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking user ID column : userid
[21:36:01] [Server thread/INFO]: [CommunityBridge] Linking player name column : username
[21:36:01] [Server thread/INFO]: [CommunityBridge] Avatars config enabled : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Post count config enabled : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Post count user ID column : null
[21:36:01] [Server thread/INFO]: [CommunityBridge] Post count post count column : null
[21:36:01] [Server thread/INFO]: [CommunityBridge] Require avatars : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Require minimum posts : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Tracking statistics : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Primary group table : user
[21:36:01] [Server thread/INFO]: [CommunityBridge] Primary group user id column : userid
[21:36:01] [Server thread/INFO]: [CommunityBridge] Primary group group id column : usergroupid
[21:36:01] [Server thread/INFO]: [CommunityBridge] Primary group uses key : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Simple synchronization enabled : true
[21:36:01] [Server thread/INFO]: [CommunityBridge] Simple synchronization direction : web-application
[21:36:01] [Server thread/INFO]: [CommunityBridge] Simple synchronization firstdirection: web-application
[21:36:01] [Server thread/INFO]: [CommunityBridge] Simple synchronization notification : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Simple synchronization P-groups : [default, player]
[21:36:01] [Server thread/INFO]: [CommunityBridge] Ban synchronization enabled : false
[21:36:01] [Server thread/INFO]: [CommunityBridge] Permissions System: PermissionsEx (PEX)
[21:36:01] [Server thread/INFO]: [CommunityBridge] SHOW TABLES LIKE 'user'
[21:36:01] [Server thread/INFO]: [CommunityBridge] SHOW COLUMNS FROM user LIKE 'userid'
[21:36:01] [Server thread/INFO]: [CommunityBridge] SHOW COLUMNS FROM user LIKE 'username'
[21:36:01] [Server thread/INFO]: [CommunityBridge] SHOW TABLES LIKE 'user'
[21:36:01] [Server thread/INFO]: [CommunityBridge] SHOW COLUMNS FROM user LIKE 'userid'
[21:36:01] [Server thread/INFO]: [CommunityBridge] SHOW COLUMNS FROM user LIKE 'usergroupid'
[21:36:01] [Server thread/INFO]: [CommunityBridge] SELECT userid FROM user WHERE userid = '1'
[21:36:01] [Server thread/INFO]: [CommunityBridge] Auto reminder started.
[21:36:01] [Server thread/INFO]: [CommunityBridge] Auto synchronization started.
[21:36:01] [Server thread/INFO]: [CommunityBridge] CommunityBridge activated.
[21:36:01] [Server thread/INFO]: [CommunityBridge] CommunityBridge is now active.
[21:36:01] [Server thread/INFO]: Server permissions file permissions.yml is empty, ignoring it
[21:36:01] [Server thread/INFO]: Done (2.421s)! For help, type "help" or "?"
[21:36:13] [User Authenticator #1/INFO]: [CommunityBridge] SELECT user.userid FROM user WHERE LOWER(username) = LOWER('GrnEyedDvl') ORDER BY userid DESC
[21:36:13] [User Authenticator #1/INFO]: [CommunityBridge] User ID '1' associated with GrnEyedDvl.
[21:36:13] [User Authenticator #1/INFO]: [CommunityBridge] GrnEyedDvl linked to web application user ID #1.
[21:36:13] [User Authenticator #1/INFO]: UUID of player GrnEyedDvl is 59ee3255-89ac-4c69-a6ab-6190234c6f22
[21:36:14] [Server thread/INFO]: GrnEyedDvl[/67.161.134.57:64619] logged in with entity id 197 at ([collab] 192.77476389565084, 66.0, 59.37688671821251)
[21:36:14] [Craft Scheduler Thread - 0/INFO]: [CommunityBridge] SELECT usergroupid FROM user WHERE userid = '1'
[21:36:29] [Async Chat Thread - #0/INFO]: didnt throw me in the Player group like it should�[m
[21:36:36] [Server thread/INFO]: GrnEyedDvl lost connection: Disconnected
[21:36:36] [Server thread/INFO]: GrnEyedDvl left the game.
[21:36:36] [Craft Scheduler Thread - 0/INFO]: [CommunityBridge] SELECT usergroupid FROM user WHERE userid = '1'
Now, here is the usergroupid that is pulled using that exact same query via command line:
mysql> select usergroupid from user where userid = '1';
+-------------+
| usergroupid |
+-------------+
| 9 |
+-------------+
In the group mapping usergroupid 9 is mated to the player group in PEX:
group-mapping:
'1' : 'default'
'2' : 'default'
'9' : 'player'
And this is what it puts in the PEX file, as you can see there is no group assigned.
users:
59ee3255-89ac-4c69-a6ab-6190234c6f22:
group: []
options:
name: GrnEyedDvl
Ok I have some more information for you. After a full reboot of the server, not just the plugin, a reboot of Ubuntu, it worked twice with two different user names. Created the entries in the PEX file in the proper groups.
You can login and out all day and the group works. Until you make a configuration change, such as manually removing the player from the list in PEX, or even changing that players group link in CB. Then it will no longer work for that player. I noticed this because my account was not getting assigned to the proper group, but when someone else came in it worked fine for him.
To replicate this, load up a new world with CB and PEX only. PEX should have at least 3 user groups in it. Login and you will get assigned to the group, like this:
[02:51:57 INFO]: [CommunityBridge] SELECT userfield.userid FROM userfield WHERE LOWER(field1) = LOWER('Remlap') ORDER BY userid DESC
[02:51:57 INFO]: [CommunityBridge] User ID '8' associated with Remlap.
[02:51:57 INFO]: [CommunityBridge] Remlap linked to web application user ID #8.
[02:51:57 INFO]: [CommunityBridge] SELECT userfield.userid FROM userfield WHERE LOWER(field1) = LOWER('taesschen') ORDER BY userid DESC
[02:51:57 INFO]: UUID of player Remlap is b6ea7af99a1c405f8fd0993395974174
[02:51:57 INFO]: UUID of player taesschen is 053606cd20a545909be463bf18773b33
[02:51:57 INFO]: Remlap[/66.253.209.140:50354] logged in with entity id 241 at ([pvp] 3263.002882251891, 74.0, 2577.2078260621647)
[02:51:57 INFO]: [CommunityBridge] SELECT displaygroupid FROM user WHERE userid = '8'
[02:51:57 INFO]: [CommunityBridge] Placed player 'Remlap' in pseudo-primary permissions group 'admin'.
Stop the world. Change the group assignment in CB. Start the world, log back in. You will not be assigned to your new group.
Stop the world. Edit the PEX file, remove your name from the list of Users. Start the world. You will not be assigned to a group, you will go to the default group.
Totally reboot the server, magically it works again. I tested this with both my custom startup script and the standard java -Xms1G -Xmx2G etc etc etc line. Both failed after shutting down the worlds and booting it up a couple of minutes later.
What I did not test is whether it behaves the same way on Bukkit that it does on Spigot. I would assume it does. I can test that tomorrow and if you want to we can set something up on TeamSpeak and I can give you access to the console and you can watch it happen live.
I did generate a new config, I just didn't list anything I didn't think was relevant. There is no place in the new file to input a table name for group-id in the player-user-linking section.
The following sections I did not change anything at all in, it doesn't apply to my setup for now, and I doubt I will ever use most of it anyways.
app-avatar-config
app-post-count-config
requirement
statistics
simple-synchronization
ban-synchronization
Since there is no place to insert the name of the group-id table and field in the main section above, it either has to go into the app-group-config section or it has to be in the multi-tables setup which isn't even in the config to begin with, or it goes someplace else that is also not documented. I saw all the examples on the main CB page, and the IPB page has the multi-tables example I used and I inserted it exactly where its inserted in that example and just renamed the tables.
http://dev.bukkit.org/bukkit-plugins/communitybridge-fm/pages/1-x-documentation/invision-power-board-config-sample/
I am more than familiar with YAML and am using the industry standard of 2 spaces per level. I just checked the file I copied information from and even though pasting into the crappy github page didn't show the spaces in the enabled: true line above the spaces are there. I guess I should figure out their bbcode or whatever they use here, but I am not on this site enough to bother with it.
Anyways, to enable group-mapping the plugin has to pull the group-id from the database someplace. What section do I need to put it in, because everything I have tried so far has failed.
The relevant sections were app-group-config and simple-synchronization. The group-mapping is only relevant to simple-synchronization and is found there; the app-group-config section focuses on the database configuration for the web-application's primary and secondary groups.
That IPB example you linked to is from the old 1.x documentation. Settings like "multi-table" don't exist any more, 2.x handles the various storage formats a different way.
I loaded up a brand new 1.6.4 world, same issue. You can replicate this.
Bukkit: git-Bukkit-1.6.4-R2.0-b2918jnks (MC: 1.6.4) (Implementing API version 1.6.4-R2.0)
CommunityBridge v2.2.2.398
PermissionsEx v1.20.4
Four groups in a VERY simple PEX file: default, player, moderator, admin
First run works.
09:12:07 [INFO] [CommunityBridge] SELECT user.userid FROM user WHERE LOWER(username) = LOWER('GrnEyedDvl') ORDER BY userid DESC
09:12:07 [INFO] [CommunityBridge] User ID '1' associated with GrnEyedDvl.
09:12:07 [INFO] [CommunityBridge] GrnEyedDvl linked to web application user ID #1.
09:12:07 [INFO] GrnEyedDvl[/67.161.134.57:55349] logged in with entity id 317 at ([world] 223.7197027661341, 70.0, 249.1362903397773)
09:12:07 [INFO] [CommunityBridge] SELECT displaygroupid FROM user WHERE userid = '1'
09:12:07 [INFO] [CommunityBridge] Placed player 'GrnEyedDvl' in pseudo-primary permissions group 'admin'.
Now, if you change the PEX file either by hand or via PEX command, then run "pex reload", then log back in, its broken, it will not stick you in the proper group. But the PEX file is fine, if I run a PEX command to put myself in a group the permissions work. CB just doesn't write to the file. You can see it grab the displaygroupid, but it doesn't run the Placed player line.
09:12:11 [INFO] test
09:12:50 [INFO] GrnEyedDvl lost connection: disconnect.quitting
09:12:50 [INFO] GrnEyedDvl left the game.
09:12:50 [INFO] [CommunityBridge] SELECT displaygroupid FROM user WHERE userid = '1'
09:12:56 [INFO] [CommunityBridge] SELECT user.userid FROM user WHERE LOWER(username) = LOWER('GrnEyedDvl') ORDER BY userid DESC
09:12:56 [INFO] [CommunityBridge] User ID '1' associated with GrnEyedDvl.
09:12:56 [INFO] [CommunityBridge] GrnEyedDvl linked to web application user ID #1.
09:12:57 [INFO] GrnEyedDvl[/67.161.134.57:55396] logged in with entity id 594 at ([world] 219.50789423393888, 69.0, 246.32457053761772)
09:12:57 [INFO] [CommunityBridge] SELECT displaygroupid FROM user WHERE userid = '1'
09:13:02 [INFO] test
09:13:09 [INFO] GrnEyedDvl lost connection: disconnect.quitting
09:13:09 [INFO] GrnEyedDvl left the game.
09:13:09 [INFO] [CommunityBridge] SELECT displaygroupid FROM user WHERE userid = '1'
pex reload
09:13:45 [INFO] Permissions file successfully reloaded
09:13:45 [INFO] Permissions reloaded
09:13:52 [INFO] [CommunityBridge] SELECTuser.useridFROMuserWHERE LOWER(username) = LOWER('GrnEyedDvl') ORDER BYuseridDESC
09:13:52 [INFO] [CommunityBridge] User ID '1' associated with GrnEyedDvl.
09:13:52 [INFO] [CommunityBridge] GrnEyedDvl linked to web application user ID #1.
09:13:52 [INFO] GrnEyedDvl[/67.161.134.57:55447] logged in with entity id 844 at ([world] 219.50789423393888, 69.0, 246.32457053761772)
09:13:52 [INFO] [CommunityBridge] SELECTdisplaygroupidFROMuserWHEREuserid= '1'
09:13:56 [INFO] test
stop
09:14:41 [INFO] CONSOLE: Stopping the server..
09:14:41 [INFO] Stopping server
Stopped the server. Load the PEX file I made before I launched the server the first time. Started the server. The only changes made were to the PEX file, I reloaded the original PEX file that worked only a few minutes ago when it added me to the admin group. This time, nothing.
09:15:44 [INFO] [CommunityBridge] SELECT user.userid FROM user WHERE LOWER(username) = LOWER('GrnEyedDvl') ORDER BY userid DESC
09:15:44 [INFO] [CommunityBridge] User ID '1' associated with GrnEyedDvl.
09:15:44 [INFO] [CommunityBridge] GrnEyedDvl linked to web application user ID #1.
09:15:44 [INFO] GrnEyedDvl[/67.161.134.57:55542] logged in with entity id 410 at ([world] 217.26935267372355, 69.0, 246.9340214796703)
09:15:44 [INFO] [CommunityBridge] SELECT displaygroupid FROM user WHERE userid = '1'
09:15:50 [INFO] test
pex user grneyeddvl
09:16:03 [INFO] 'GrnEyedDvl' is a member of:
09:16:03 [INFO] default (rank 1000 @ default)
09:16:03 [INFO] GrnEyedDvl's permissions:
09:16:03 [INFO]
09:16:03 [INFO] GrnEyedDvl's options:
Its caching something someplace, and is not updating it on server stop and start.
Ok I was not seeing the Players folder before, but obviously its storing stuff about players in there. The problem with that is that when you are editing stuff by command line you are not browsing the folders, you just call up the file and edit it. I had no idea you were caching stuff now, and there is no real reason to at all. It just makes it more work to set stuff up because every time you make a change you now have to go delete a player file.
Shame on me for not seeing the where you are caching this and wasting your time reading the top part of this thread. But the way you have this setup makes absolutely no sense at all to begin with and is the main source of the problem, I just wasn't seeing it that way.
The reason you cache data, whether its for this MC plugin or for a forum is to save overhead. You don't have to run a query again, or rewrite a file again. Its already there. But here is the problem.
You are not saving any overhead by caching these results. Even if you have 1000 players, it really should only be 1000 queries, and that is assuming that every single player logs in every single day. A standard MySQL server can easily handle an extra 1000 queries a day, my small (Minecraft) server does something like 250,000 queries a day anyways. My big one at Total War Center does something like 2 million a day. 1000 individual player queries is nothing.
But here is the even bigger problem. YOU ARE STILL RUNNING THE QUERIES! You are not saving anything by caching the results. If you were checking the cached files first and then running the query if it didn't find a player file I would understand it. I wouldn't agree with it but it would work. It would still be a disk read but not a MySQL query
The plugin still runs the query as it shows up in the log, so you are not saving the query. It seems like you are trying to save the write cycle, which is the wrong way to do it unless you are going to check a few more things.
You are running the query, then comparing the query results to the cache file. So if the config file has changed, it doesn't matter because you are comparing it to cached data and not to the actual configuration. This is why players do not update when you move stuff around.
You cannot cache data like this permanently. Even DNS records and DHCP records and memcache results and every other cache you have ever heard of have a TTL, and a method of updating them when needed even before the TTL expires. Your cache does not update at all. At least not on my setup.
My suggestion is that you dump the cached Player files entirely. Now understanding that you are doing some write info back to the database if people have it configured that way there may actually be an argument for keeping it and not holding that data in RAM for however long they are on the server. But you are still running the groupid query on exit, so you aren't saving anything there, you are just adding an extra step by reading the cache file and comparing it to the config. (at least I assume you are, haven't looked at the code) instead of reading it from RAM.
If you are going to keep the cache files, then they HAVE to update on player login.
That is not a cache. Think about what it means to synchronize in two directions and how to detect changes on each side in a meaningful way.
- An ideal support request includes a copy of the configuration file (with the database information redacted), and version numbers for the permissions plugin, CommunityBridge, and the Minecraft Server.
- PEX will return "default" as a group membership when the player in fact has no memberships. Its recommended that "default" as a group name as that makes the results murky: are we seeing "default" because it is a for real group, or because PEX returns "default" when it should return nothing?
- There is no need to run the plugin as root unless there is something about your firewall settings that prevents non-root processes from opening MySQL connections (which seems unlikely).
- The data stored in the Players directory takes a bit of explanation. For CommunityBridge to know which side actually changed when there is a difference between the game server and the web application, it needs to know what each side "looked like" the last time it attempted a synchronization. So it stores a snapshot of the state of both sides after completing a synchronization. If a synchronization occurs while the configuration isn't quite right, it can be the case that the recorded information is incorrect. In that situation, the best thing to do is delete the plugins/CommunityBridge/Players directory and let it treat all players as if it had never seen them before.