depstar

Discussion around https://github.com/seancorfield/depstar
fabrao 2021-02-12T18:29:05.082600Z

hello all. How do I rund depstar to only include .class and not .clj in jar?

seancorfield 2021-02-12T18:29:29.083100Z

Sounds like you want the :exclude regex option?

fabrao 2021-02-12T18:29:38.083300Z

even when using uberjar

seancorfield 2021-02-12T18:30:08.083800Z

See https://github.com/seancorfield/depstar/#excluding-files

fabrao 2021-02-12T18:30:19.084Z

why is it including .clj in libs ?

fabrao 2021-02-12T18:30:26.084300Z

not .class

seancorfield 2021-02-12T18:31:01.085Z

Not all .clj files get compiled (Clojure itself has at least one .clj file that doesn't get compiled).

seancorfield 2021-02-12T18:32:14.085500Z

(just checked, Clojure 1.10.2 has 43 .clj files)

fabrao 2021-02-12T18:32:46.086400Z

so, I can not provide a jar with only .class?

seancorfield 2021-02-12T18:34:00.087800Z

Even if you AOT your main namespace with depstar, anything that is not statically reachable from it will not be compiled: AOT is purely transitive. If you use :compile :all, it will attempt to compile all of your own namespaces -- not just your main ns -- but even that won't necessarily reach all possible .clj files in all the transitive dependencies.

fabrao 2021-02-12T18:35:26.088700Z

yes, I used it :compile :all but I saw that other libs did not been compile

fabrao 2021-02-12T18:36:29.089500Z

what is the matter about not compile everything into .class?

seancorfield 2021-02-12T18:37:53.089900Z

I just explained that.

fabrao 2021-02-12T18:38:12.090100Z

shame me, sorry

fabrao 2021-02-12T18:40:39.093500Z

what you normaly use? :compile :all ?

seancorfield 2021-02-12T18:41:16.094600Z

:compile :all will compile all the namespaces in your project, which will also transitively compile every namespace that is statically reachable from your code: that's how Clojure's compile function works. If there are namespaces in libraries you use that are not statically reachable from any of your namespaces, they will not be "seen" by compile and they will not be compiled to .class files. That means if you use :exclude to omit all .class files, you may fail at runtime if a namespace that was not statically reachable is then loaded dynamically. You should be able to :exclude just your own code's .clj files from the JAR if you're careful with the regex -- and you do not have any code that is only reachable dynamically.

seancorfield 2021-02-12T18:42:16.095800Z

At work, we just use :aot true :main-class <some ns>. We "don't care" if source ends up in the JAR and we "don't care" about the potential "runtime compilation" overhead for namespaces that are only loaded dynamically.

fabrao 2021-02-12T18:42:56.096300Z

so, no way to exist an obfuscator to clojure?

seancorfield 2021-02-12T18:44:28.097800Z

Like I keep saying -- and as it explains in the section of the README that I linked to: you can use :exclude with a regex to omit .clj files -- but you need to be careful to not exclude something that ends up being dynamically loaded at runtime because those would not be compiled by Clojure (because they are not statically reachable).

seancorfield 2021-02-12T18:46:04.099Z

And it's also worth pointing out that JVM bytecode decompilers exist: so even if your code is all .class files, folks can decompile that into Java source code for the most part. It's weird Java source code but it's going to allow anyone to figure out how your code works.

seancorfield 2021-02-12T18:46:13.099300Z

What problem are you trying to solve here?

fabrao 2021-02-12T18:49:44.100900Z

I didn´t know that in .jar it´s carrying the .clj and someone can see it. The .class , I think, it´s more difficult to read it

fabrao 2021-02-12T18:50:24.101900Z

I thought it was compiled all into .class stuffs

seancorfield 2021-02-12T18:50:52.102400Z

Nearly all Clojure libraries are source code. It's actually bad to compile a library since it can cause version conflicts later on. Compiling an uberjar/application is fine but all it does is improve startup time.

fabrao 2021-02-12T18:52:01.103Z

I got it. So, what happen when you compile to GraalVM?

seancorfield 2021-02-12T18:52:22.103200Z

No idea, never used it.

fabrao 2021-02-12T18:53:17.103700Z

thanks for your explanation

seancorfield 2021-02-12T19:03:42.107500Z

As a concrete example:

(! 685)-> clojure -X:new :template app :name fabrao/example
Generating a project called example based on the 'app' template.
(! 686)-> cd example/
(! 687)-> cat deps.edn 
...
  :uberjar {:replace-deps {seancorfield/depstar {:mvn/version "2.0.171"}}
            :exec-fn hf.depstar/uberjar
            :exec-args {:aot true
                        :jar "example.jar"
                        :main-class "fabrao.example"
                        :sync-pom true}}}}
;; specify compile all instead of default behavior:
(! 688)-> clojure -X:uberjar :compile :all
[main] INFO hf.depstar.uberjar - Synchronizing pom.xml
Skipping paths: resources
[main] INFO hf.depstar.uberjar - Compiling fabrao.example ...
[main] INFO hf.depstar.uberjar - Building uber jar: example.jar
[main] INFO hf.depstar.uberjar - Processing pom.xml for {fabrao/example {:mvn/version "0.1.0-SNAPSHOT"}}
;; note that there's only one ns in this project so compile all is no different that just aot'ing that one (main) namespace!
(! 689)-> jar tvf example.jar |fgrep .clj
   253 Fri Feb 12 10:56:51 PST 2021 fabrao/example.clj
 12711 Tue Jan 26 12:54:48 PST 2021 clojure/string.clj
... and about 40 more .clj files from Clojure itself ...
(! 690)-> jar tvf example.jar |fgrep fabrao
     0 Fri Feb 12 10:57:19 PST 2021 fabrao/
  1439 Fri Feb 12 10:57:19 PST 2021 fabrao/example$_main.class
  1340 Fri Feb 12 10:57:19 PST 2021 fabrao/example$fn__140.class
  1916 Fri Feb 12 10:57:19 PST 2021 fabrao/example$greet.class
  1581 Fri Feb 12 10:57:19 PST 2021 fabrao/example$loading__6738__auto____138.class
  1964 Fri Feb 12 10:57:19 PST 2021 fabrao/example.class
  3046 Fri Feb 12 10:57:19 PST 2021 fabrao/example__init.class
   253 Fri Feb 12 10:56:51 PST 2021 fabrao/example.clj
     0 Fri Feb 12 10:57:19 PST 2021 META-INF/maven/fabrao/
     0 Fri Feb 12 10:57:19 PST 2021 META-INF/maven/fabrao/example/
   109 Fri Feb 12 10:57:19 PST 2021 META-INF/maven/fabrao/example/pom.properties
  1594 Fri Feb 12 10:57:19 PST 2021 META-INF/maven/fabrao/example/pom.xml
So fabrao.example is present in both source and compiled form. We can exclude that source:
(! 694)-> clojure -X:uberjar :compile :all :exclude '["fabrao.*\\.clj"]'
[main] INFO hf.depstar.uberjar - Synchronizing pom.xml
Skipping paths: resources
[main] INFO hf.depstar.uberjar - Compiling fabrao.example ...
[main] INFO hf.depstar.uberjar - Building uber jar: example.jar
[main] INFO hf.depstar.uberjar - Processing pom.xml for {fabrao/example {:mvn/version "0.1.0-SNAPSHOT"}}
(! 695)-> jar tvf example.jar |fgrep fabrao
     0 Fri Feb 12 11:02:45 PST 2021 fabrao/
  1439 Fri Feb 12 11:02:45 PST 2021 fabrao/example$_main.class
  1340 Fri Feb 12 11:02:45 PST 2021 fabrao/example$fn__140.class
  1916 Fri Feb 12 11:02:45 PST 2021 fabrao/example$greet.class
  1581 Fri Feb 12 11:02:45 PST 2021 fabrao/example$loading__6738__auto____138.class
  1964 Fri Feb 12 11:02:45 PST 2021 fabrao/example.class
  3046 Fri Feb 12 11:02:45 PST 2021 fabrao/example__init.class
     0 Fri Feb 12 11:02:46 PST 2021 META-INF/maven/fabrao/
     0 Fri Feb 12 11:02:46 PST 2021 META-INF/maven/fabrao/example/
   109 Fri Feb 12 11:02:46 PST 2021 META-INF/maven/fabrao/example/pom.properties
  1594 Fri Feb 12 11:02:46 PST 2021 META-INF/maven/fabrao/example/pom.xml
No more fabrao source files (but you'll still get the 40+ Clojure source files).