Clojurians Log v2

Can someone please explain to me what Shaun Lebron might have been thinking about with this?

https://youtu.be/K0Tsa3smr1w?t=2054

(grey ... (white) ...
  (green ... (lt-green (gold) ...) 
    (blue)))

I guess my question is more about why he thinks the depth view is more meaningful than the nesting view

I think it's only when the open expression is on the left, that you indent?

and you don't with the ones in the middle? or maybe that's obvious, not sure

I’m using test-runner (https://github.com/cognitect-labs/test-runner) for testing and made a github action for it, but it somehow constantly fails with

Syntax error (FileNotFoundException) compiling at (farmmorning/case_format_test.clj:1:1).
Could not locate farmmorning/util/case_format__init.class, farmmorning/util/case_format.clj or farmmorning/util/case_format.cljc on classpath. Please check that namespaces with dashes use underscores in the Clojure file name.

here is the deps configuration.

:aliases {:dev     {:extra-paths ["dev/src"]}
           :test    {:extra-paths ["test"]
                     :extra-deps {io.github.cognitect-labs/test-runner
                                  {:git/url "<https://github.com/cognitect-labs/test-runner.git>"
                                   :sha "705ad25bbf0228b1c38d0244a36001c2987d7337"}}
                     :exec-fn cognitect.test-runner.api/test}}

seems like malli channel is kinda dead. asking for help

dead like in no-one answering in 20min? :thinking_face:

yes

Please wait for at least a few hours before judging a channel as "dead" and reposting a question in a different one.

By that definition, #clojure seems to be more dead

Hi all, I ran across Alex Miller’s article “Improving Development Startup Time” (https://clojure.org/guides/dev_startup_time). That seems like something I would want 🙂 I’m having trouble getting it to work however. When I execute (compile …) I only see the class files of that namespace, not the ones required by it. Whereas the article says “transitive” which I take to mean that all namespaces it depends on should also be compiled. So I compile a few of them by hand, do a jack-in, and start my dev server, and it isn’t one bit faster. Now that may of course be due to other reasons, I didn’t profile the startup process, but I’d like to make sure that I’m not missing something. I added “dev” and “classes” to extra-paths in deps.edn, I even added :jvm-opts ["-Dclojure.compile.path=classes"]. Any idea if I’m doing something wrong?

> When I execute (compile …) I only see the class files of that namespace, not the ones required by it That's not what I'm seeing when I compile the top level ns of an ns tree

@stefan.van.den.oord

borkdude@MBP2019 ~/Dropbox/dev/clojure/babashka (master) $ mkdir classes
borkdude@MBP2019 ~/Dropbox/dev/clojure/babashka (master) $ ls classes
borkdude@MBP2019 ~/Dropbox/dev/clojure/babashka (master) $ clj
Clojure 1.11.0-alpha1
user=> (compile 'babashka.main)
babashka.main
user=>
borkdude@MBP2019 ~/Dropbox/dev/clojure/babashka (master) $ ls classes
babashka    bencode     borkdude    cheshire    clj_yaml    clojure     cognitect   edamame     flatland    hf          hiccup      org         rewrite_clj sci         selmer

Hi, I find a performance gap between the rand-int clojure function and the java native one for instance .nextIn of java.util.Random. The gap is small for small amount of iterations but increasing with the number of iterations. @alexmiller,I saw some of your posts asking for feedback on performance, so I ask if I did something wrong, or if youo think this is something which should be improved in the clojure standard library, or maybe my case is so specific that I have to use interop?

(ns poc.defrecord.slack)

(def ^java.util.Random seed (java.util.Random.))
(defn compare-rands [nb-iter]
  [{:method :java-call
    :iterations nb-iter
    :time (with-out-str  (time
                          (dotimes [_ nb-iter]
                            (.nextInt seed 100))))}
   {:method :native-rand
    :iterations nb-iter
    :time (with-out-str (time (dotimes [_ nb-iter]
                                (rand-int 100))))}])

(compare-rands 2000000)
;; => [{:method :java-call,
;;      :iterations 2000000,
;;      :time "\"Elapsed time: 35.315238 msecs\"\n"}
;;     {:method :native-rand,
;;      :iterations 2000000,
;;      :time "\"Elapsed time: 127.318862 msecs\"\n"}]
;; => [{:method :java-call,
;;      :iterations 200,
;;      :time "\"Elapsed time: 0.017836 msecs\"\n"}
;;     {:method :native-rand,
;;      :iterations 200,
;;      :time "\"Elapsed time: 0.025678 msecs\"\n"}]

In my original code there are two call to rand-intwhich seems to increase the influence:

(ns poc.defrecord.slack)

(def ^java.util.Random seed (java.util.Random.))
(defn compare-rands [nb-iter]
  [{:method :java-call
    :iterations nb-iter
    :time (with-out-str  (time
                          (dotimes [_ nb-iter]
                            (.nextInt seed 100)
                            (.nextInt seed 100)
                            (.nextInt seed 100))))}
   {:method :native-rand
    :iterations nb-iter
    :time (with-out-str (time (dotimes [_ nb-iter]
                                (rand-int 100)
                                (rand-int 100)
                                (rand-int 100))))}])

(compare-rands 200)
;; => [{:method :java-call,
;;      :iterations 200,
;;      :time "\"Elapsed time: 0.011242 msecs\"\n"}
;;     {:method :native-rand,
;;      :iterations 200,
;;      :time "\"Elapsed time: 0.039123 msecs\"\n"}]
;; => [{:method :java-call,
;;      :iterations 2000000,
;;      :time "\"Elapsed time: 93.312348 msecs\"\n"}
;;     {:method :native-rand,
;;      :iterations 2000000,
;;      :time "\"Elapsed time: 372.539035 msecs\"\n"}]

Strange. Thanks for checking!

rand-int calls Random/nextDouble under the hood, and does some mutiplication and casting to get an int

so some overhead is expected

Using time for benchmarks is not very reliable btw

I suggest using https://github.com/hugoduncan/criterium

Ok I will try criterium. It's true the time are not stable

Concerning the implementation your remark is an explanation but it may be possible to do better ?

I looked at the implementation it is quite straightforward

implementing rand-int using Random.nextInt(int bound) might be faster

skipping the overhead of conversion

You mean the double to int conversion?

yes

also Random.nextDouble looks like it does (a little) more work than Random.nextInt

but probably the overhead is not that large

Yes random functions are based on natural integer series.

So generating double is more time consuming

I know the difference is small but in my case it is doing a difference

if you a processing on multiple threads then using ThreadLocalRandom might also speed things up

I am not currently but it is im my plan so I take the advice

rand-int and rand use Math/random(), which uses a shared java.util.Random instance for all threads, which may cause contention

I'm trying out :replace-deps with juxt/pack.alpha.git like https://github.com/juxt/pack.alpha/pull/92/files, I consistently get a few seconds slower run-times, (I run once to warmup and then average over three runs). Is that expected?

this:

:pack {:extra-deps {pack/pack.alpha {:git/url "<https://github.com/juxt/pack.alpha.git>"
                                                :sha "042c61410b1851957ae5f77cfe26a255837172cc"}}
                  :main-opts ["-m"]}

:pack {:replace-deps {pack/pack.alpha {:git/url "<https://github.com/juxt/pack.alpha.git>"
                                                :sha "042c61410b1851957ae5f77cfe26a255837172cc"}}
                  :main-opts ["-m"]}

I thought it would be faster since it would load less code, but maybe I'm misunderstanding it?

The first time you run it will compute and cache classpaths so make sure you’re accounting for that

I would generally have the same expectation as you - replace should replace the project deps so include less

You can use -Stree or -Spath to confirm that

when I benchmarked I ran once to "warmup" and averaging 3 runs.

I'll take alook at -Stree/-Spath

actually, after killing all other applications on my laptop the results look better

once all that stuff is downloaded and cached, there should be no real difference here - in both cases you're just running the program

aha, so the main difference is when there is no cache?

yeah, it's just a question of how many deps you have to download

but once you download them to your .m2/.gitlibs ... that's all cached

What is a good place to look for templates for clj-new?

is there anyway to prevent a symbol from being unmapped via ns-unmap ?

no?

why do you want to do that?

I don’t. Someone asked me if it could and I was almost certain that it couldn’t but wanted to check anyway.

The person who asked me is worried about malicious code. Of course, this person hasn’t discovered eval yet 😄

powerful tools are also often dangerous :)

Yes, of course. What is interesting is that truly dynamic systems have to allow for the change of their own environments/source, and everything from DNA to running code is susceptible to maliciousness

the converse is also true. Only dynamic safety is real safety. For example, unchecked array bounds have been a massively costly impact of trusting static assumptions over dynamic reality ...

@srijayanth if you want to have a kind of sandbox that forbids certain vars, you could take a look at sci

user=> (sci/eval-string "(ns-unmap *ns* 'inc)" {:deny ['ns-unmap]})
Execution error (ExceptionInfo) at sci.impl.utils/throw-error-with-location (utils.cljc:51).
ns-unmap is not allowed!

clojail is another approach which relies on java sandboxing

How do a type hint a vararg call for an array of Foo objects? I know to call with (into-array Foo [])

however I try it I seem to get a reflection warning 😕

I’ve tried variants of things like (a and b are already type hinted) (.method a b ^"baz.bar.Foo" (into-array Foo [])

^"[Lbaz.bar.Foo;"

See (type (into-array Foo [])) for the string representation of the type.

ah sorry that’s already what I have… I misstyped it here

👀

hmm prints correctly

Perhaps there’s some other aspect of ambiguity in the call then?

maybe the issue is something else

Does the reflection warning provide a bit more specific information?

ok real code is:

(def default-context ^"[Lorg.eclipse.rdf4j.model.Resource;" (into-array Resource []))

(defn index-data! [input-files]
  (let [repo (repo/sail-repo (repo/memory-store))]
    (doseq [^File input input-files]
      (with-open [^RepositoryConnection conn (repo/->connection repo)]
        (let [^RDFFormat fmt (.orElse (Rio/getParserFormatForFileName (str input)) nil)]
          (.add conn input "<http://ignoreable>" fmt default-context))))
    repo))

swirrl/qb_tools/concept_scheme/positions.clj:89:11 - call to method add on org.eclipse.rdf4j.repository.RepositoryConnection can't be resolved (argument types: java.io.File, java.lang.String, org.eclipse.rdf4j.rio.RDFFormat, unknown).

class I’m calling is: https://rdf4j.org/javadoc/3.0.4/org/eclipse/rdf4j/repository/RepositoryConnection.html#add-java.io.File-java.lang.String-org.eclipse.rdf4j.rio.RDFFormat-org.eclipse.rdf4j.model.Resource...-

hmm actually the docs are one minor version out, from what I have on my classpath :thinking_face:

Maybe the type hint on the def has to be before the symbol instead of on the value?

ahh that’s it thanks

dev=> (def example ^"[Ljava.lang.String;" (into-array String []))
#'dev/example
dev=> (meta example)
nil
dev=> 

silly mistake

6pm code blindness setting in 😩

Is there anyone here who provides hands-on introductory Clojure workshops for corporate teams, or can recommend such from personal experience?

Hadn’t Cognitect used to offer that? @alexmiller?

I can't personally recommend him, but I like the output of Eric Normand and it sounds like he does that sort of thing: https://lispcast.com/training/

https://clojure.org/community/training

^^ resources

you can ping <mailto:sales@cognitect.com|sales@cognitect.com> - we are entertaining such requests but I don't know the full status