Redirector [Modern]

Redirector [Modern]

27M Downloads

一种完美修复现在以及未来可能出现的所有兼容性问题的方案(至少在我两百多个模组的1.20.1整合里没问题,而之前是有问题的)

roj234 opened this issue · 5 comments

commented

简单说法:限制这个优化仅作用于直接for-each循环
例如 for(var x : XXX.values())
代码见下(实测大概排除了1/3的函数,这毕竟只是一个简单的模式匹配,要怪就怪模组开发者)
在我的包里:yes=1166, no=461, total=377807

比如它没法识别的安全模式:Enum.values()[random.nextInt(Enum.values().length)]
但是如果把values存到一个本地变量里就能识别了
还有一些该死的lambda导致的识别问题,不过因为这个是白名单,最多误杀,不会放过有问题的

匹配器的核心代码:
注意,我是自己设计的ASM框架,和大家用的不一定兼容
但是整体逻辑可以参考
此外这个优化是需要处理两遍的,所以需要一个持久化的配置文件,这次启动MC时进行采样,下次进行优化

配置实体

    public static class RedirectorConfig {
        @Comment("启用本模块")
        public boolean enable = true;
        @Comment("检测或应用模式\n注意,在初次启动时会自动进行分析")
        public boolean detectionMode = true;
        @Comment("找到的枚举类型定义")
        public Set<String> declaration = new HashSet<>();
        @Comment("找到的枚举类型引用")
        public Map<String, Set<MemberDescriptor>> reference = new HashMap<>();

        public transient Set<String> enumReference = new HashSet<>();

        public void end() {
            enumReference.removeIf(type -> !declaration.contains(type));
            declaration.removeIf(type -> !enumReference.contains(type));
        }
    }
···

配置逻辑
···java
        if (INSTANCE.redirector.enable) {
            if (INSTANCE.redirector.detectionMode) {
                INSTANCE.redirector.declaration.clear();
                INSTANCE.redirector.reference.clear();
                Redirector.Find transformer = new Redirector.Find(INSTANCE.redirector);
                CoremodLoader.喜加一(transformer);
                Runtime.getRuntime().addShutdownHook(new Thread(() -> {
                    System.out.println("summaries: yes="+transformer.totalApply+", no="+transformer.totalSkip+", total="+transformer.noMatch);
                    INSTANCE.redirector.detectionMode = false;
                    INSTANCE.redirector.end();
                    save();
                }));
            } else {
                CoremodLoader.喜加一(new Redirector(INSTANCE.redirector));
            }
        }

转换器类

class Redirector implements Transformer {
	private final ModuleConfig.RedirectorConfig config;

	public Redirector(ModuleConfig.RedirectorConfig config) {this.config = config;}

	@Override
	public boolean transform(String name, Context ctx) throws TransformException {
		var changed = false;
		if (config.declaration.contains(name)) {
			ClassNode data = ctx.getData();
			for (MethodNode method : data.methods) {
				if (method.name().equals("values") && method.rawDesc().startsWith("()[L")) {
					if (data.getMethodObj("redirector$values") != null) return false;

					var node = method.getAttribute(data.cp, Attribute.Code).instructions.getNodeAt(0);

					var redirector$values = data.newMethod(Opcodes.ACC_STATIC | Opcodes.ACC_PUBLIC, "redirector$values", method.rawDesc());
					redirector$values.visitSize(1, 0);
					redirector$values.field(Opcodes.GETFIELD, node.desc());
					redirector$values.insn(Opcodes.ARETURN);
					redirector$values.finish();
					changed = true;
					break;
				}
			}
		}
		if (config.reference.containsKey(name)) {
			var descriptors = config.reference.get(name);
			ClassNode data = ctx.getData();
			for (MethodNode method : data.methods) {
				var methodRef = new MemberDescriptor("", method.name(), method.rawDesc());
				if (descriptors.contains(methodRef)) {
					System.out.println("applying to "+methodRef);
					Code code = method.getAttribute(data.cp, Attribute.Code);
					for (InsnNode node : code.instructions) {
						if (node.opcode() == Opcodes.INVOKESTATIC) {
							MemberDescriptor desc = node.desc();
							if (this.config.declaration.contains(desc.owner) && desc.name.equals("values") && desc.rawDesc.startsWith("()[L")) {
								desc.name = "redirector$values";
								break;
							}
						}
					}
				}
			}
			return true;
		}
		return changed;
	}

	/** 第一遍Transformer */
	public static class Find implements Transformer {
		int totalApply, totalSkip, noMatch;
		private final ModuleConfig.RedirectorConfig config;

		public Find(ModuleConfig.RedirectorConfig config) {
			this.config = config;
		}

		@Override
		public synchronized boolean transform(String name, Context ctx) throws TransformException {
			ClassNode data = ctx.getData();
			if (data.parent().equals("java/lang/Enum")) {
				config.declaration.add(data.name());
			}

			for (MethodNode method : data.methods) {
				Code code = method.getAttribute(data.cp, Attribute.Code);
				/*if (code != null) for (InsnNode node : code.instructions) {
					if (node.opcode() == Opcodes.AASTORE) break;
					if (node.opcode() == Opcodes.INVOKESTATIC) {
						MemberDescriptor desc = node.desc();
						if (desc.name.equals("values") && desc.rawDesc.startsWith("()[L")) {
							List<MemberDescriptor> list = config.reference.computeIfAbsent(data.name(), Helpers.fnArrayList());
							MemberDescriptor md = new MemberDescriptor(method.owner(), method.name(), method.rawDesc());
							if (!list.contains(md)) list.add(md);
							break;
						}
					}
				}*/
				if (code != null) {
					var ok = mayApply(method, code, data);
					if (ok == null) {
						totalApply++;
						System.out.println("apply for "+method.copyDesc());
					} else if (!ok.equals("no match")) {
						totalSkip++;
						System.out.println("skip for"+method+", because of "+ok);
					} else {
						noMatch++;
					}
				}
			}
			return false;
		}

		private static final boolean moreBogus = false;
		private String mayApply(MethodNode method, Code code, ClassNode data) {
			InsnNode node = code.instructions.getNodeAt(0);
			var enumTypes = new HashSet<String>();
			while (node != null) {
				if (node.opcode() == Opcodes.INVOKESTATIC) {
					MemberDescriptor desc = node.desc();
					safe:
					if (desc.name.equals("values") && desc.rawDesc.startsWith("()[L")) {
						enumTypes.add(desc.owner);

						node = node.next();
						String immediateOp = Opcodes.toString(node.opcode());
						if (immediateOp.equals("ArrayLength")) break safe;
						if (immediateOp.startsWith("ILoad") || node.getAsInteger() != null) {
							node = node.next();
							if (Opcodes.toString(node.opcode()).equals("AALoad")) continue;
							else return "ldc+!aaload";
						}
						if (immediateOp.equals("InvokeStatic")) {
							MemberDescriptor desc1 = node.desc();
							if (desc1.owner.equals("java/util/Arrays") && desc1.name.equals("stream")) break safe;
							if (desc1.owner.equals("java/util/stream/Stream") && desc1.name.equals("of")) break safe;
						}
						// allow putstatic, since nearly everyone stores it will not modify them (to cache)
						if (moreBogus && immediateOp.startsWith("PutStatic")) break safe;
						if (!immediateOp.startsWith("AStore")) return "invokestatic+!astore";

						int slot = node.getVarId();
						// after store
						var node1 = node;
						while (true) {
							node1 = node1.next();
							if (node1 == null) break;

							check:
							if (slot == node1.getVarId()) {
								String op = Opcodes.toString(node1.opcode());
								// ok, this part is safe……
								if (op.startsWith("AStore")) break;
								if (!op.startsWith("ALoad")) return "unknown instruction "+node1;

								node1 = node1.next();
								if (node1.opcode() == Opcodes.ARRAYLENGTH) break check;
								if (Opcodes.toString(node1.opcode()).startsWith("ILoad") || node1.getAsInteger() != null) {
									node1 = node1.next();
									if (Opcodes.toString(node1.opcode()).equals("AALoad")) continue;
									return "excepting aaload not ldc+"+node1;
								}
								// allow change variable slot, but since we only track one slot, old might be changed……
								if (true && Opcodes.toString(node1.opcode()).startsWith("AAStore")) {
									slot = node1.getVarId();
									continue;
								}
								return "excepting ldc not "+node1;
							}
						}

						break;
					}
				}

				node = node.next();
			}

			if (!enumTypes.isEmpty()) {
				config.enumReference.addAll(enumTypes);
				var methodRef = new MemberDescriptor("", method.name(), method.rawDesc());
				var whitelist = config.reference.computeIfAbsent(data.name(), Helpers.fnHashSet());
				whitelist.add(methodRef);
				return null;
			}

			return "no match";
		}
	}
}

感谢你看到最后,我相信你会问上了这么多代码为什么不发PR
您看这代码和这模组兼容吗,我连模组编译工具链都是自己做的而不使用gradle,你让我怎么兼容(战术后仰

commented

当然可以,你怎么做都好,我的只是提出一个建议
至于负优化,在我看来仅仅一次的额外开销还算合理,实测也没有增加很多的初次启动时间
另外我之所以使用白名单,而不是黑名单,就是为了不需要主动检测模组修改然后重新采样数据
就怎么说呢,我觉得这依然是一种优化…… 不过,至少你回复的很及时,在我的issue体验中算是很好的一次购物(bushi

commented

行吧上面的代码有问题没跑起来,要把GETFIELD换成GETSTATIC,然后还和一堆优化模组冲突,好吧,但是我感觉这个思路是没问题的……

commented

哦,然后还要禁止把java自带的枚举统计进去……你使用forge的API应该不会有这个问题,我注入的类加载器可能太底层了
还要排除FML的DistMarker,还有禁用lithium的三个优化
//"alloc.enum_values.living_entity.LivingEntityMixin",
//"alloc.enum_values.piston_handler.PistonHandlerMixin",
//"alloc.enum_values.redstone_wire.RedstoneWireBlockMixin",
,modernfix的一个
//"perf.cache_blockstate_cache_arrays.AbstractBlockStateCacheMixin",
,这四个都是同样改枚举的

不管怎么样,这(指基于白名单之后)完全是免费的性能提升

commented

我的意见是检测,先打开debug模式试跑整合包,redirector检测enum的修改情况,将异常enum打在log里加入config屏蔽。
这样只需要在打开debug模式时多插入一个hook,检查enum[]有没有被打乱即可。

commented

这个效果,你看如何?或者我把你的设计加入为“安全模式”,

[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: java.lang.IllegalStateException
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at net.minecraft.client.shader.ShaderLoader$ShaderType.redirectionor_logError(ShaderLoader.java)
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at net.minecraft.client.shader.ShaderLoader$ShaderType.redirectionor_checkEnums(ShaderLoader.java)
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at net.minecraft.client.shader.ShaderLoader$ShaderType.values(ShaderLoader.java:97)
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at com.Hileb.teampotato.redirectionor.demo.EnumTestDemo.enumFacing(EnumTestDemo.java:19)
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at com.Hileb.teampotato.redirectionor.demo.EnumTestDemo.onLaunch(EnumTestDemo.java:13)
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[15:25:57] [main/INFO] [STDERR]: [com.Hileb.teampotato.redirectionor.Redirectionor:logError:30]: 	at java.lang.reflect.Method.invoke(Method.java:498)

但我觉得真正安全的是用 Mixin 对特定目标。
同时,这个模组的优化点在减小不必要的 EnumArray#clone() ,这个开销并没有大到需要一个十分复杂的逻辑支撑,否则会造成负优化。