graalvm

Discuss GraalVM related topics. Use clojure 1.10.2 or newer for all new projects. Contribute to https://github.com/clj-easy/graal-docs and https://github.com/BrunoBonacci/graalvm-clojure. GraalVM slack: https://www.graalvm.org/slack-invitation/.
ericdallo 2021-01-27T12:47:21.031900Z

I kind of found the issue, I made a simple reify of the LanguageServer from lsp4j :

(defn fake-server []
  (reify LanguageServer
    (^CompletableFuture initialize [_this ^InitializeParams params]
     (do
       (println "INITIALIZING!")
       (log/info "INITIALIZING!")))

    (getTextDocumentService [_this]
      nil)
    (getWorkspaceService [_this]
      nil)))
During the start of the app, I call:
(log/info "-->" (class server)
                    (:members (r/reflect server)))
That prints all methods from the class, it works perfect with lein run,but with graalvm it prints #{}

borkdude 2021-01-27T12:59:43.032100Z

So what does this tell you?

ericdallo 2021-01-27T13:01:55.032300Z

It means the for some reason, graalvm is not working with that reify

borkdude 2021-01-27T13:04:17.032800Z

@ericdallo what error do you get when using it?

ericdallo 2021-01-27T13:05:43.033Z

Jan 27, 2021 9:46:32 AM org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint request
WARNING: Unsupported request method: initialize

ericdallo 2021-01-27T13:06:23.033200Z

because it can't find the implementation of one of the methods initialize

borkdude 2021-01-27T13:06:49.033400Z

because you haven't reified it?

borkdude 2021-01-27T13:07:01.033600Z

oh sorry I misread

ericdallo 2021-01-27T13:07:46.033800Z

this is what I get from lein run / java from graalvm home:

--> clojure_lsp.main$fake_server$reify__31521 #{#clojure.reflect.Method{:name initialize, :return-type java.util.concurrent.CompletableFuture, :declaring-class clojure_lsp.main$fake_server$reify__31521, :parameter-types [org.eclipse.lsp4j.InitializeParams], :exception-types [], :flags #{:public}} #clojure.reflect.Method{:name getTextDocumentService, :return-type org.eclipse.lsp4j.services.TextDocumentService, :declaring-class clojure_lsp.main$fake_server$reify__31521, :parameter-types [], :exception-types [], :flags #{:public}} #clojure.reflect.Field{:name const__4, :type java.lang.Object, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:public :static :final}} #clojure.reflect.Method{:name meta, :return-type clojure.lang.IPersistentMap, :declaring-class clojure_lsp.main$fake_server$reify__31521, :parameter-types [], :exception-types [], :flags #{:public}} #clojure.reflect.Field{:name const__2, :type clojure.lang.Var, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:public :static :final}} #clojure.reflect.Method{:name getWorkspaceService, :return-type org.eclipse.lsp4j.services.WorkspaceService, :declaring-class clojure_lsp.main$fake_server$reify__31521, :parameter-types [], :exception-types [], :flags #{:public}} #clojure.reflect.Field{:name const__1, :type clojure.lang.Var, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:public :static :final}} #clojure.reflect.Field{:name const__6, :type clojure.lang.Keyword, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:public :static :final}} #clojure.reflect.Method{:name withMeta, :return-type clojure.lang.IObj, :declaring-class clojure_lsp.main$fake_server$reify__31521, :parameter-types [clojure.lang.IPersistentMap], :exception-types [], :flags #{:public}} #clojure.reflect.Constructor{:name clojure_lsp.main$fake_server$reify__31521, :declaring-class clojure_lsp.main$fake_server$reify__31521, :parameter-types [clojure.lang.IPersistentMap], :exception-types [], :flags #{:public}} #clojure.reflect.Constructor{:name clojure_lsp.main$fake_server$reify__31521, :declaring-class clojure_lsp.main$fake_server$reify__31521, :parameter-types [], :exception-types [], :flags #{:public}} #clojure.reflect.Field{:name const__3, :type clojure.lang.Keyword, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:public :static :final}} #clojure.reflect.Field{:name const__0, :type clojure.lang.Var, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:public :static :final}} #clojure.reflect.Field{:name const__5, :type clojure.lang.Keyword, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:public :static :final}} #clojure.reflect.Field{:name __meta, :type clojure.lang.IPersistentMap, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:final}} #clojure.reflect.Field{:name const__7, :type java.lang.Object, :declaring-class clojure_lsp.main$fake_server$reify__31521, :flags #{:public :static :final}}}
It's printing correct all methods from that reify

borkdude 2021-01-27T13:08:09.034Z

@ericdallo Maybe this will work: Is it possible to make the reified object at the top-level, at compile time?

ericdallo 2021-01-27T13:08:46.034200Z

yes it's possible, changing from defn to def would be enough?

borkdude 2021-01-27T13:09:30.034400Z

think so

ericdallo 2021-01-27T13:09:38.034600Z

Let me try it :loading:

ericdallo 2021-01-27T13:23:41.034800Z

Same issue, this is the output of that print from above:

--> class clojure_lsp.main$reify__31521 #{}

borkdude 2021-01-27T13:24:15.035Z

also the same error when used?

ericdallo 2021-01-27T13:25:07.035200Z

yep, maybe try with gen-class? https://stackoverflow.com/a/8614783

borkdude 2021-01-27T13:30:48.035600Z

or add the class to the reflection config

Shantanu Kumar 2021-01-27T13:31:54.035800Z

@ericdallo Maybe try once without type-hinting the initialize method? Since it’s the only method in the interface, it should work without hinting due to auto-inference.

ericdallo 2021-01-27T13:31:55.036Z

But how can I get the class name? or you mean add the interface to the reflection config?

ericdallo 2021-01-27T13:32:34.036200Z

good point @kumarshantanu, I'll need type hint for other methods later, but for that test it's valid, let me try

borkdude 2021-01-27T13:32:38.036400Z

yeah, the interface

1πŸ‘
ericdallo 2021-01-27T13:44:06.036700Z

It gives another error, but it seems is a step ahead πŸ˜†

SEVERE: Unable to invoke no-args constructor for class org.eclipse.lsp4j.adapters.InitializeParamsTypeAdapter$Factory. Registering an InstanceCreator with Gson for this type may fix this problem.
java.lang.RuntimeException: Unable to invoke no-args constructor for class org.eclipse.lsp4j.adapters.InitializeParamsTypeAdapter$Factory. Registering an InstanceCreator with Gson for this type may fix this problem.
	at com.google.gson.internal.ConstructorConstructor$14.construct(ConstructorConstructor.java:226)
	at com.google.gson.internal.bind.JsonAdapterAnnotationTypeAdapterFactory.getTypeAdapter(JsonAdapterAnnotationTypeAdapterFactory.java:55)
	at com.google.gson.internal.bind.JsonAdapterAnnotationTypeAdapterFactory.create(JsonAdapterAnnotationTypeAdapterFactory.java:49)
	at com.google.gson.Gson.getAdapter(Gson.java:423)
	at com.google.gson.Gson.fromJson(Gson.java:887)
	at org.eclipse.lsp4j.jsonrpc.json.adapters.MessageTypeAdapter.fromJson(MessageTypeAdapter.java:329)
	at org.eclipse.lsp4j.jsonrpc.json.adapters.MessageTypeAdapter.parseParams(MessageTypeAdapter.java:249)
	at org.eclipse.lsp4j.jsonrpc.json.adapters.MessageTypeAdapter.read(MessageTypeAdapter.java:119)
	at org.eclipse.lsp4j.jsonrpc.json.adapters.MessageTypeAdapter.read(MessageTypeAdapter.java:55)
	at com.google.gson.Gson.fromJson(Gson.java:888)
	at org.eclipse.lsp4j.jsonrpc.json.MessageJsonHandler.parseMessage(MessageJsonHandler.java:119)
	at org.eclipse.lsp4j.jsonrpc.json.MessageJsonHandler.parseMessage(MessageJsonHandler.java:114)
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.handleMessage(StreamMessageProducer.java:193)
	at org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen(StreamMessageProducer.java:94)
	at org.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run(ConcurrentMessageProcessor.java:113)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.lang.Thread.run(Thread.java:834)
	at com.oracle.svm.core.thread.JavaThreads.threadStartRoutine(JavaThreads.java:519)
	at com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine(PosixJavaThreads.java:192)
Caused by: java.lang.reflect.InvocationTargetException
	at java.lang.reflect.Method.invoke(Method.java:566)
	at com.google.gson.internal.UnsafeAllocator$1.newInstance(UnsafeAllocator.java:50)
	at com.google.gson.internal.ConstructorConstructor$14.construct(ConstructorConstructor.java:223)
	... 21 more
Caused by: java.lang.IllegalArgumentException: Class org.eclipse.lsp4j.adapters.InitializeParamsTypeAdapter$Factory is instantiated reflectively but was never registered. Register the class by using org.graalvm.nativeimage.hosted.RuntimeReflection
	at com.oracle.svm.core.graal.snippets.SubstrateAllocationSnippets.hubErrorStub(SubstrateAllocationSnippets.java:247)
	at sun.misc.Unsafe.allocateInstance(Unsafe.java:840)
	... 24 more


Process clojure-lsp stderr<1> finished

ericdallo 2021-01-27T13:44:27.036900Z

Not sure is related to remove the type hinting or to move to the reflection.json

borkdude 2021-01-27T13:44:41.037100Z

never change two things at a time ;)

ericdallo 2021-01-27T13:44:59.037300Z

yeah hahaha

ericdallo 2021-01-27T13:45:15.037500Z

it seems another issue now, it's serializing the json to respond to client

ericdallo 2021-01-27T13:45:31.037700Z

Probably I should do something to initialize Gson correctly

ericdallo 2021-01-27T13:46:21.037900Z

Oh this looks a good start:

Class org.eclipse.lsp4j.adapters.InitializeParamsTypeAdapter$Factory is instantiated reflectively but was never registered. Register the class by using 

ericdallo 2021-01-27T13:46:36.038100Z

should I add it to reflection.json?

borkdude 2021-01-27T13:46:44.038300Z

guess so

1
ericdallo 2021-01-27T13:49:23.038600Z

If that works, probably I'll need to add https://github.com/eclipse/lsp4j/tree/master/org.eclipse.lsp4j/src/main/xtend-gen/org/eclipse/lsp4j/adapters to the reflection too, if that works, I'll certainly open an issue on lsp4j with steps how to make it work with graalvm 🀞

ericdallo 2021-01-27T14:50:04.038800Z

Good news, problem solved with the suggestion from you, @borkdude , I added some classes to reflection json and it's getting into the method initialize correctly now βœ… The next issue πŸ˜† is: clojure-lsp tries to parse Gson json objects via https://github.com/clojure-lsp/clojure-lsp/blob/master/src/clojure_lsp/interop.clj#L416-L432, and it's returning this error on runtime with graal:

com.oracle.svm.core.posix.thread.PosixJavaThreads.pthreadStartRoutine            PosixJavaThreads.java:  192                                                                
                              com.oracle.svm.core.thread.JavaThreads.threadStartRoutine                 JavaThreads.java:  519                                                                
                                                                   java.lang.Thread.run                      Thread.java:  834                                                                
                                     java.util.concurrent.ThreadPoolExecutor$Worker.run          ThreadPoolExecutor.java:  628                                                                
                                      java.util.concurrent.ThreadPoolExecutor.runWorker          ThreadPoolExecutor.java: 1128                                                                
                                                    java.util.concurrent.FutureTask.run                  FutureTask.java:  264                                                                
                                    java.util.concurrent.Executors$RunnableAdapter.call                   Executors.java:  515
                          org.eclipse.lsp4j.jsonrpc.json.ConcurrentMessageProcessor.run  ConcurrentMessageProcessor.java:  113
                            org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.listen       StreamMessageProducer.java:   94
                     org.eclipse.lsp4j.jsonrpc.json.StreamMessageProducer.handleMessage       StreamMessageProducer.java:  194
                                       org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.consume              RemoteEndpoint.java:  190
                                 org.eclipse.lsp4j.jsonrpc.RemoteEndpoint.handleRequest              RemoteEndpoint.java:  261
                             org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.request             GenericEndpoint.java:  120
                       org.eclipse.lsp4j.jsonrpc.services.GenericEndpoint.lambda$null$0             GenericEndpoint.java:   65
                                                                                    ...                                       
clojure_lsp.main.proxy$clojure_lsp.ClojureExtensions$LanguageServer$c8d5825a.initialize                                       
                                                                 clojure-lsp.main/fn/fn                         main.clj:  326
                                                       clojure-lsp.main/client-settings                         main.clj:  278
                                                          clojure-lsp.interop/json->clj                      interop.clj:  421
                                                                                    ...                                       
java.lang.IllegalArgumentException: No matching field found: isJsonNull for class com.google.gson.JsonObject

ericdallo 2021-01-27T14:50:34.039200Z

Should I add com.google.gson.JsonObject to reflection too?

borkdude 2021-01-27T15:36:56.039500Z

yes

borkdude 2021-01-27T15:37:07.039700Z

at least that method

ericdallo 2021-01-27T15:41:14.039900Z

After that I get:

java.lang.NoSuchMethodError: java.lang.reflect.AccessibleObject.canAccess(java.lang.Object)
Not sure what to add to reflection.json now

borkdude 2021-01-27T15:42:26.040100Z

Maybe java.lang.reflect.AccessibleObject πŸ˜†

ericdallo 2021-01-27T15:42:46.040300Z

Yeah, that was my guess πŸ˜› compiling :loading:

borkdude 2021-01-27T15:43:04.040500Z

you can also run with a graalvm agent and then it will spit out all the things it needed reflection for during the run of a program

borkdude 2021-01-27T15:43:15.040700Z

it will also spit out a lot of classes you don't need, but it might give you some idea

borkdude 2021-01-27T15:43:23.040900Z

be sure to run with direct linking, this might help

ericdallo 2021-01-27T15:44:40.041100Z

Yeah, I thought about that, but the issue is that is not trivial to test clojure-lsp on terminal, we need to send LSP spec jsons, I'll try to move it to a script that use the java from graalvm, hope it works

borkdude 2021-01-27T15:47:33.041300Z

I mean, this is just a setting when creating the uberjar

borkdude 2021-01-27T15:47:43.041500Z

-J-Dclojure.compiler.direct-linking=true

borkdude 2021-01-27T15:47:53.041700Z

oh

borkdude 2021-01-27T15:47:58.041900Z

I see, never mind

ericdallo 2021-01-27T15:51:33.042100Z

np, thanks for the tip πŸ™‚

ericdallo 2021-01-27T18:36:18.042500Z

Having this now:

Can't call public method of non-public class: public sun.nio.fs.UnixPath sun.nio.fs.UnixPath.getParent()
even with this on reflection.json:
{
        "name": "sun.nio.fs.UnixPath",
        "allDeclaredConstructors": true,
        "allPublicConstructors": true,
        "allDeclaredMethods": true,
        "allPublicMethods": true
    }
Related to this line: https://github.com/clojure-lsp/clojure-lsp/blob/master/src/clojure_lsp/crawler.clj#L199

Shantanu Kumar 2021-01-27T19:24:28.042900Z

@ericdallo Try hinting dir as a public interface? (.getParent ^java.nio.file.Path dir)

ericdallo 2021-01-27T19:25:12.043100Z

Hum, good guess again, let me try it, I was thinking in refactor that to use <http://clojure.java.io|clojure.java.io> instead of manually using java classes, similar how clj-kondo do to get the config dir

borkdude 2021-01-27T19:27:28.043400Z

@ericdallo You should never have to include OS specific things in your reflection config, this config won't work on Windows for example

borkdude 2021-01-27T19:28:12.043900Z

either for the return type or for the local

ericdallo 2021-01-27T19:30:51.044100Z

You are right, something like that?

[dir ^java.nio.file.Path  (shared/uri-&gt;path project-root)]
@borkdude

borkdude 2021-01-27T19:31:21.044400Z

yes, but if you place this type hint on your function it would be even better

1πŸ‘1βž•