closure() instanceof closure should return true, but for whatever reason, it causes a compiler error, Unexpected string type passed to "instanceof". instanceof(closure(), closure) works, however, so this is probably related to the keyword resolution.

Implementing instanceof as a keyword might not offer a convenient solution for this problem. I would propose to handle instanceof as an operator (CSymbol) instead, using __autoconcat__.rewrite(...) to handle instanceof the same as for example + and *. This also automatically makes instanceof left-associative, which can be changed to non-associative in instanceof()'s postParseRewrite(...) if desired.

The exception in the last comment is caused by optimizeDynamic() receiving a different number of children/arguments than functions accept. It's possible to get around that by checking that in each optimizeDynamic() implementation, but we should probably just not optimize functions which are going to throw a compile error for a mismatching argument size anyways.

Hmm, yes, that should work. In general though, it might be nice to be able to do this with the best of both worlds, that is, implement a "late binding keyword" or whatever, where the keyword is marked as operating on the resolved left hand, right hand, both, or neither. This would also probably be where we implement return as a keyword as well. It would also continue to fit nicely into the ecosystem around syntax highlighting and such.

Fixed here: 938a9f5

I believe it's because the instanceof keyword is consuming the left and right hand nodes, which skips the keyword processing for the right hand as processKeywords() continues to iterate through the nodes. So closure remains a CKeyword. Normally keywords would be processed as children, but closure is not a child of instanceof at this stage. Perhaps InstanceofKeyword should call processKeywords() for the right hand side before consuming it.

I believe it's because the instanceof keyword is consuming the left and right hand nodes, which skips the keyword processing for the right hand as processKeywords() continues to iterate through the nodes. So closure remains a CKeyword. Normally keywords would be processed as children, but closure is not a child of instanceof at this stage. Perhaps InstanceofKeyword should call processKeywords() for the right hand side before consuming it.

You make a good point. My initial thought here is that resolving the right side of instanceof first might make it right associative, where I expected it to be left associative now (e.g. a instanceof b instanceof c being executed as (a instanceof b) instanceof c, where handling the right side would convert that to a instanceof (b instanceof c)). But it appears that it's either non-associative or broken.

(true instanceof boolean) instanceof boolean // true

true instanceof boolean instanceof boolean // COMPILE ERROR: Unexpected keyword: instanceof

true instanceof (boolean instanceof boolean) // Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 1, Size: 1

It should be given some thought as to which associativity should be used here (non-associative would probably make sense) and how to enforce that. When that's decided, it should be added to the operators table in the documentation Other than that, I do believe that your solution would work.

Additional test:

1 instanceof int // success
1 instanceof (int) // Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 1, Size: 1

Full trace:

Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
        at java.util.ArrayList.rangeCheck(ArrayList.java:659)
        at java.util.ArrayList.get(ArrayList.java:435)
        at com.laytonsmith.core.functions.DataHandling$_instanceof.optimizeDynamic(DataHandling.java:3708)
        at com.laytonsmith.core.MethodScriptCompiler.optimize(MethodScriptCompiler.java:2467)
        at com.laytonsmith.core.MethodScriptCompiler.optimize(MethodScriptCompiler.java:2362)
        at com.laytonsmith.core.MethodScriptCompiler.compile(MethodScriptCompiler.java:1847)
        at com.laytonsmith.tools.Interpreter.execute(Interpreter.java:782)
        at com.laytonsmith.tools.Interpreter.execute(Interpreter.java:717)
        at com.laytonsmith.tools.Interpreter.textLine(Interpreter.java:481)
        at com.laytonsmith.tools.Interpreter.<init>(Interpreter.java:318)
        at com.laytonsmith.tools.Interpreter.<init>(Interpreter.java:239)
        at com.laytonsmith.core.Main$InterpreterMode.execute(Main.java:544)
        at com.laytonsmith.core.Main.main(Main.java:313)