Clojurians Log v2

Hi everyone, Some wierd problem is happening in our project. Inside our resources folder we have some sample configuration file called om.edn . Then in another project the former's artifact is imported. Whenever I give it an absolute path for some configuration - I am using the aero config library and I give the master config's absolute path, inside which lies a #include "./om.edn" , but instead of loading the intended config file the sample om.edn file from the original repository is loaded. What's happening ? Is there something about jar files that I do not know ? Thanks alot

Hi all, I'm working on a clj library for my job that we were looking at using as a lein plugin. We're not entirely sure if we should be using this as a plug-in at all, but the general idea is that we want to generate code from database schemas, and a connection needs to be supplied as a map. If it's advisable to do this, what's the best way to parse the arguments supplied from the console in the defn for the leiningen command? My intuition was supply the relevant function from the lib and give it the args as a coll using (apply fn args) .

@ho0man it sounds like you maybe have just a clash on the classpath. If both your dependency artifact and your own have the same file name in same spot it could clash. It’s more of an aero config lib question but I believe it defaults to looking for classpath relative resources. Not sure what you had is actually an “absolute” path

hmm ... thanks @mikerod Think I have to read a little bit more about classpath resources

@edward.hughes1911 It doesn’t sound like a plugin to me. I don’t see what about that tasks involves project-meta-level work that plugins provide. It seems like you should just make an application with a -main entry point that is able to take args from the command line to do the work you are saying. I may be missing some broader context around what you are wanting to use this for though. Are you trying to use this common setup from multiple projects? You could supply this tool as a dependency to them and then call something to do the building or use something like lein-shell and/or :prep-tasks in those consuming projects.

I had a thought that it would be easier to just call it as a library dependency and run it at the REPL to generate the code, but we were instructed explore the plug-in route. The concept for the plugin is that it will provide functions to generate clojure specs for databases, and is then called via lein from the project directory that will be used as a central library for those specs. db connection maps and targets for the generated code are to be passed as plugin task options in the project.clj of the spec library.

Hey all, lein uberjar is stuck after upgrading clojure to 1.10 from 1.8 and a few other dependencies

What could be the cause of this?

do a thread dump (ctrl-\) or with lein it might be a second process so you can jps -mv and probably figure out the pid, then do jstack <pid>

Thanks for the quick reply Alex, I can't seem to understand why the "main" thread is in waiting state, any idea why from a quick glance?

"main" #1 prio=5 os_prio=31 tid=0x00007fd6b4800800 nid=0xe03 in Object.wait() [0x0000700006668000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x000000077c675778> (a java.lang.Thread)
	at java.lang.Thread.join(Thread.java:1252)
	- locked <0x000000077c675778> (a java.lang.Thread)
	at java.lang.Thread.join(Thread.java:1326)
	at <http://com.hypirion.io|com.hypirion.io>.Pipe.join(Pipe.java:120)
	at leiningen.core.eval$sh.invokeStatic(eval.clj:191)
	at leiningen.core.eval$sh.doInvoke(eval.clj:173)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:646)
	at clojure.core$apply.invoke(core.clj:641)
	at leiningen.core.eval$fn__5732.invokeStatic(eval.clj:262)
	at leiningen.core.eval$fn__5732.invoke(eval.clj:260)
	at clojure.lang.MultiFn.invoke(MultiFn.java:233)
	at leiningen.core.eval$eval_in_project.invokeStatic(eval.clj:366)
	at leiningen.core.eval$eval_in_project.invoke(eval.clj:356)
	at leiningen.core.eval$eval_in_project.invokeStatic(eval.clj:360)
	at leiningen.core.eval$eval_in_project.invoke(eval.clj:356)
	at leiningen.compile$compile.invokeStatic(compile.clj:166)
	at leiningen.compile$compile.invoke(compile.clj:135)
	at clojure.lang.Var.invoke(Var.java:379)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.core$apply.invokeStatic(core.clj:648)
	at clojure.core$apply.invoke(core.clj:641)
	at leiningen.core.main$partial_task$fn__5932.doInvoke(main.clj:272)
	at clojure.lang.RestFn.invoke(RestFn.java:410)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:29)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:648)
	at clojure.core$apply.invoke(core.clj:641)
	at leiningen.core.main$apply_task.invokeStatic(main.clj:322)
	at leiningen.core.main$apply_task.invoke(main.clj:308)
	at leiningen.core.eval$run_prep_tasks.invokeStatic(eval.clj:51)
	at leiningen.core.eval$run_prep_tasks.invoke(eval.clj:43)
	at leiningen.core.eval$prep.invokeStatic(eval.clj:86)
	at leiningen.core.eval$prep.invoke(eval.clj:73)
	at leiningen.jar$build_jar.invokeStatic(jar.clj:283)
	at leiningen.jar$build_jar.invoke(jar.clj:280)
	at leiningen.jar$main_jar.invokeStatic(jar.clj:292)
	at leiningen.jar$main_jar.invoke(jar.clj:288)
	at leiningen.jar$jar.invokeStatic(jar.clj:343)
	at leiningen.jar$jar.invoke(jar.clj:325)
	at leiningen.uberjar$uberjar$fn__8098.invoke(uberjar.clj:167)
	at leiningen.uberjar$uberjar.invokeStatic(uberjar.clj:167)
	at leiningen.uberjar$uberjar.invoke(uberjar.clj:143)
	at leiningen.uberjar$uberjar.invokeStatic(uberjar.clj:187)
	at leiningen.uberjar$uberjar.invoke(uberjar.clj:143)
	at clojure.lang.Var.invoke(Var.java:379)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.core$apply.invokeStatic(core.clj:648)
	at clojure.core$apply.invoke(core.clj:641)
	at leiningen.core.main$partial_task$fn__5932.doInvoke(main.clj:272)
	at clojure.lang.RestFn.invoke(RestFn.java:410)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.RestFn.applyTo(RestFn.java:132)
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:29)
	at clojure.lang.RestFn.applyTo(RestFn.java:137)
	at clojure.core$apply.invokeStatic(core.clj:648)
	at clojure.core$apply.invoke(core.clj:641)
	at leiningen.core.main$apply_task.invokeStatic(main.clj:322)
	at leiningen.core.main$apply_task.invoke(main.clj:308)
	at leiningen.core.main$resolve_and_apply.invokeStatic(main.clj:328)
	at leiningen.core.main$resolve_and_apply.invoke(main.clj:324)
	at leiningen.core.main$_main$fn__5998.invoke(main.clj:401)
	at leiningen.core.main$_main.invokeStatic(main.clj:394)
	at leiningen.core.main$_main.doInvoke(main.clj:391)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.lang.Var.invoke(Var.java:379)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.core$apply.invokeStatic(core.clj:646)
	at clojure.main$main_opt.invokeStatic(main.clj:314)
	at clojure.main$main_opt.invoke(main.clj:310)
	at clojure.main$main.invokeStatic(main.clj:421)
	at clojure.main$main.doInvoke(main.clj:384)
	at clojure.lang.RestFn.invoke(RestFn.java:436)
	at clojure.lang.Var.invoke(Var.java:388)
	at clojure.lang.AFn.applyToHelper(AFn.java:160)
	at clojure.lang.Var.applyTo(Var.java:700)
	at clojure.main.main(main.java:37)

I don't have any top level form and the main function takes a command as arg and based on the command it calls the corresponding function.

Moreover it used to work before the upgrade of clojure and some other dependencies

are you sure this is the right pid?

I'm no expert on leiningen but I think the eval-in-project is going to launch a 2nd vm and here it's just waiting for that to complete

I see "prep-task" in the stack trace, and a shell call

so it's not the clojure compile, but some shell call that is done before compiling, looks like

isn't uberjar task in the stack there?

the actual compiling would happen in the second jvm with the needed classpath etc

yes, but the shell out is higher in that trace

oh - yeah that

that compile doesn't actually shell out, it's the prep-task code that is shelling out unless I'm reading the stack wrong

Oops sorry it was the uberjar process' dump

"Hashed wheel timer #1" #14 prio=5 os_prio=31 tid=0x00007fddbc395800 nid=0x3a07 waiting on condition [0x000070000d820000]
   java.lang.Thread.State: TIMED_WAITING (sleeping)
	at java.lang.Thread.sleep(Native Method)
	at org.jboss.netty.util.HashedWheelTimer$Worker.waitForNextTick(HashedWheelTimer.java:445)
	at org.jboss.netty.util.HashedWheelTimer$Worker.run(HashedWheelTimer.java:364)
	at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
	at java.lang.Thread.run(Thread.java:748)

This is the only non-daemon thread

and it is in TIMED_WAITING

that method looks like a watcher of some sort - something that is polling some state(?)

it shouldn't be something that keeps the vm running though

I'm guessing maybe you updated jetty?

sorry, netty?

I'd start backing off those dep updates and see if you can find a smaller working delta and proceed incrementally

it would be surprising to me if the Clojure bump was the cause vs a dep

Yes I think a twitter api dep, updated netty

will start doing what you suggested

usually it's something like a top-level def that should be delayed etc

so that compile doesn't actually start threads, launch missiles, etc

Are requires compiled too before the main actually uses any of them?

yes

I'd check if some twitter API usage of a netty thread pool is getting initialzied in a def

Thanks a lot for the pointers, I will work on it

It was indeed a def spawning a twitter client. Thanks a lot!

I still just don’t see any capabilities this would need that are related to being a plugin - as in running in the same process as lein - seems more of a liability than a benefit when you don’t need it