Clojurians Log v2

to add one to the already mentioned: Eastwood maybe I'd call it "dynamic analysis" though because it works by evaling your code under tools.analyzer. In exchange one gets great accuracy

Had a fun time last night tweaking vim - like emacs, tweaking vim is an excellent place to waste hours getting ready to be productive, 10/10, highly recommended :rolling_on_the_floor_laughing: Anyway. I decided to setup Conjure, having had slightly more time with vim than last I tried it, everything made more sense and in general, the plugin is just fabulous (saying this in case its creator still lurks in here). However, in my case, I actually use CoC (conqueror of completion), a rather popular plugin for “intellisense” - code navigation, autocompletion and the like, and I encountered a bit of a problem: Using clojure-lsp I can have code navigation and refactoring tips from konda(?) etc. OR I can use coc-conjure to get the auto-completion and suggestions from the cider nrepl through which Conjure connects fed into coc — but not both..! My work-around has been to cram in deoplete, only use this for clojure and to fight with coc to disable any auto-suggestion/auto-complete functionality (just for clojure), in this way, I use CoC as a driver of the clojure-lsp process while deoplete gets its completion information through Conjure which in turn gets it from Cider. This seems hack’ish, but aside from some similar setup, e.g. another LSP plugin and then using CoC for completion, I cannot see what one should do in this case. Any input? Did I miss something stupidly obvious ? Curious to hear.

Kibit as well

Don't know the answer, but you might have better luck getting help in #conjure or in the Conjure Discord (https://conjure.fun).

Hi all, I’m trying to find an old blog post that I only vaguely remember from years back - it was a deep dive into laziness, and a very technical, detailed post. Pretty sure it was written by one of the early clojure luminaries (e.g. someone like fogus, cemerick, technomancy, ztellman), but I’ve checked their blogs etc and can’t seem to find it. I know it wasn’t stuart sierra. I seem to remember it covering the subtleties around maximal laziness, mapcat etc… With lots of good examples comparing the differences… though to be honest I can’t really remember much about the topics other than it was to do with sequences. I know I’ll recognise it when I see it though 😆 Can anyone suggest where it might be??

maybe this one - https://stuartsierra.com/2015/04/26/clojure-donts-concat it is about StackOverflow in concat

Nope not that one, I know it definitely wasn’t by stuart sierra 🙂

It was far more detailed than that

I vaguely remember it having hand drawn illustrations in it; but could be wrong about that.

Is there a simple way to "hook" a function so that it runs every time a form in evaluated, regardless of namespace? For example, I want to instrument core/hfun so that runs when I reevaluate either core/foo or impl/bar.

you might want to have a look at this library https://github.com/alexanderjamesking/spy it implements similar idea but for testing maybe its source code can give you a hint

Is that what motfrom is asking for? It sounds like he’s wanting to hook eval itself, not the functions. In which case you can do it by customising your repl. What repl are you using?

For the eval itself, yes. I'm currently using nrepl, but I could switch to a socket repl if that makes things easier.

or you could just wrap the defn in a do that calls the required function?

see here: https://insideclojure.org/2020/02/11/custom-repl/

this is for the default clojure repl; not nREPL

nREPL you can do it with a middleware… maybe other ways too

Ah, cool! That's fine, I have no experience hacking nrepl but i guess it would be a middleware there

oh, I didn’t catch that this should be a property of the repl which is not the only way to run clojure programs

yeah it’s certainly worth asking what your use case is. Hooking eval in the repl is only going to work in dev; though you probably don’t want to do it anywhere else 🙂

no problem! my usecase is a simple static site generator for use in the repl. The idea is that an eval could trigger the build-function. Not as nice as Oz-like live reload, but good enough for what I'm doing and would not require any (cl)js.

which also means that yes, it is purely for development use : ) Doing that in any sort of production env would be... interesing

i "enjoyed" the top heading in this repo https://github.com/technomancy/robert-hooke (obviously a different thing, but still)

I’m not convinced eval granularity is going to align with wanting to rebuild a site, but knock yourself out 🙂

No, that's a fair point. I'm not sure either.

It sounds like a bad idea to me

rebuilding is trivial so time is not the issue, but it would be nice to be able to control rebuilds to certains namespaces, like the one I'm using for garden

and for some templates

You could just call rebuild yourself at the REPL… or set up a filesystem watch to rebuild on save

Personally I’d be tempted to do something like sticking a rebuild function in user.clj and just run (user/rebuild) which will then work from any namespace when you want to rebuild.

that's what I'm doing right now, calling the rebuild and then refreshing the browser with a watch. but calling that rebuild requires navigating to a different file and navigating to that form, which adds some friction when you are doing things like editing css or smaller template changes

use an editor hotkey to send the form to the repl

having it in user is not a bad idea. idealy, one could make an emacs keybind to send that eval (user/rebuild) in the repl without having to be on in that specific place

that's what I'm doing right now, calling the rebuild and then refreshing the browser with a watch. but calling that rebuild requires navigating to a different file and navigating to that form, which adds some friction when you are doing things like editing css or smaller template changes

(messages should be in the reverse order)

Anyone familiar with speclj ? I am getting this error after setting up a new project with lein new speclj and then running lein test.

Exception in thread "main" java.io.FileNotFoundException: Could not locate speclj__init.class or speclj.clj on classpath., compiling:(clojure_functions/reduce_spec.clj:1:1)

I was thinking that Aphyr might have posted about this, but I can't find anything in particular. Here's one about sequences, which mentions lazyness: https://aphyr.com/posts/304-clojure-from-the-ground-up-sequences

(I appreciate that these oldie-but-goldie posts get some new light)

Thanks, but unfortunately that’s not the one… I think it was more about the implementation and edge cases, rather than how to use the library.

could you check if your project.clj has speclj declared as dependency for dev profile?

my project.clj: (_defproject_ clojure-functions "0.1.0-SNAPSHOT"   _:description_ "FIXME: write description"   _:url_ "<http://example.com/FIXME>"   _:license_ {_:name_ "Eclipse Public License"             _:url_ "<http://www.eclipse.org/legal/epl-v10.html>"}   _:main_ clojure-functions.core   _:dependencies_ [[org.clojure/clojure "1.8.0"]]   _:profiles_ {_:dev_ {_:dependencies_ [[speclj "3.3.2"]]}}   _:plugins_ [[speclj "3.3.2"]]   _:test-paths_ ["spec"])

did you run lein test or lein spec ?

lein test

https://github.com/slagyr/speclj#with-leiningen

you should use lein spec instead I think this is the most convenient way to run specs

Ok! I guess I dont know the difference between spec and test 😃

lein test is an option of leiningen itself speclj introduce one more option to run itself

Yes, and lein test is dependant on the :main NS file in the project file. After changing that to the correct (from clojure-functions.core to clojure.functions.reduce) the lein test gave me the correct error I think

Well well I got it working now, thanks a lot 😃 I should have read the documentation more thoroughly, I have a tendency to expect things to just work a certain way

that is ok ) there are no persons who knows everything) I also learned that difference just now)

Will this always be true? That no new map is created when a value is updated to an identical one?

(def m {:a 42})

(identical? m
            (assoc m :a 42))

yes

sorry, I misread, no I would not guarantee that

I would guarantee they are equal

but whether they are identical is an implementation question

Right, so when remaining 'identical' is important, it is best to get and assoc if needed rather than using update (same applies to xxx-in variants)

I'd say it's bad to rely on identity in the first place

and then you don't care

In the grand scheme of things yes. I am wondering about some possible optimizations for a project but otherwise I agree this would be too much of a burden.

(update m :key identity) does not return identical result on records btw @adam678

current dev environment results from 2021 Clojure survey - still time to register your own choices! https://www.surveymonkey.com/r/clojure2021

Hi, please advice how I can split byte-array into two byte-arrays (based on index) basically i want to split ouput of this

(defn random-bytes
  [size]
  (let [seed (byte-array size)]
    (.nextBytes (SecureRandom.) seed)
    seed))

You can create two new byte-array objects (which are the same as Java arrays with primitive type byte for the elements) with the lengths of the two pieces you want, and copy the parts of the original array into those two new arrays. If this is a hot inner loop, you can do the copying via the JVM method call System/arrayCopy. You could also write a loop in Clojure over the indexes of the array you want to copy, and copy the bytes one at a time in that loop.

also see newer java.util.Arrays

Don't think I'd used the methods of that class before. copyOfRange looks useful for this purpose.

Clojure homebrew install says 1.10.2.774 but run says 1.10.1 details in thread.

hash -d clj

then try again

> ➜ ~ hash -d clj > hash: no such directory name: clj

clojure version might be overwritten by local deps.edn check if you have it in $HOME/.clojure/deps.edn

clojure version is whatever dep you've declared

new 1.10.2.x wil ldefault to 1.10.2 in root, but you can override in user or project

you can use any clojure tool version with any clojure version

@claudius.nicolae Just as a follow-up to that, if you look at my user-level deps.edn file, you'll see I have aliases for every version of Clojure all the way back to 1.0 (and three different aliases for the specific sub-versions of Clojure 1.10): https://github.com/seancorfield/dot-clojure/blob/develop/deps.edn#L171-L188

That can be very useful for testing code against several different versions of Clojure, via the CLI.

Hello! I'm not sure if here is the right place, and if isn't I'm sorry in advance, but can anybody help me to understand a problem? I have this function:

(ns core.reader
  (:require [<http://clojure.java.io|clojure.java.io> :as io]))

(defn read-lines
  "reads an input and returns a ISeq of the lines"
  [input]
  (line-seq (io/reader input)))

(ns core.reader-test
  (:require [core.reader :as reader]
            [clojure.test :refer :all]))

(deftest core-reader-test
  (testing "read from stdin"
    (let [result (with-in-str (reader/read-lines *in*) "is\n42")]
      (is (= "is" (first result)))
      (is (= "42" (last result))))))

@matheusashton have you tried adding a final newline?

hmmm no, I'll try

@matheusashton never mind- you need a doall, or else the read-lines executes outside the with-in-str body

laziness plus dynamic bindings are a bad mix

but that still isn't unblocking it...

@matheusashton also you have the args of with-in-str flipped

I’d also make a my own output stream and not muck about with “in”

(cmd)user=> (with-in-str "is\n42" (doall (read-lines *in*)))
("is" "42")
(cmd)user=> (with-in-str "is\n42" (read-lines *in*))
Error printing return value (IOException) at java.io.PushbackReader/ensureOpen (PushbackReader.java:73).

@dpsutton inputstream, surely

Yeah. I always get the terms backwards. I always forget if output stream has output or accepts output

@dpsutton my mental model is that everything is from the perspective of the object as a first person entity

if you can call a read method, it's an input stream

That makes sense. I think I put myself as the consumer and think “I need output from something. Ah a stream”

But I’ll probably remember it now so thanks

but that's projecting! it's your input

it's someone else's output

also there are things like StringReader that are really nobody's "output" per se, just a view of their data as if it were

Sounds like a proverb: "one man's output is another man's input."

@matheusashton I think this is simpler than what you are doing currently

user=> (read-lines (java.io.StringReader. "is\n42"))
("is" "42")

@noisesmith thanks! I'll try to fix the things that you've pointed, the input of my program will be the contents of a file (I cannot read the file inside the program because I don't know the filename) so I need to read it from *in* , I was trying to make like an integration test, that's why I tried to dynamic bind *in* , I run my program like lein run < file

@matheusashton my example above makes it read from a string, you don't need to send a string to in, then read in, you can just read from the string

maybe use with-in-str if you need to test -main, but usually I'd make sure -main is trivial and test the individual components in ways that don't require process level resources

yes, but I was trying to make an integration test, simulating the real behaviour

StringReader is as real as *in* is, and also simpler

hmm ok

in this instance, it's simpler because *in* changes value as your object changes scope, and thanks to laziness your result would change scope before being realized if you didn't force it

as a big picture example, I'd set things up like this: (defn app [stream] ...) (defn -main [& args] (app *in*))

where "app" has all the logic aside from implementation details (Iike which specific reader you are reading lines from)

now app can be tested without relying on global / process level state like *in*

ok, got it!

Thank you very much!

this pattern makes testing much easier (but it's only as good as the quality of your stubs - swapping one reader for another is great, your hand made stub for your protocol might be iffier...)

sure

also, my -main is like this:

(defn -main
  [& args]
  (->> *in*
       (read-lines)
       (map parse-json)
       (map println)))

parse-json is simply cheshire's parse-string

right, nothing is consuming the return value of map, so why would it calculate anything?

perhaps you want (run! println) instead at the end

oh right, it doesn't realize the whole pipe until something needs right?

@matheusashton right, because every function in the pipeline is lazy, it doesn't promise to calculate any more results than are consumed, and in the case of -main when not run from a repl, nobody consumes it

the repl consumes it in order to print (and would give you a big annoying list of nils from each println call)

ok, thanks! Learned a lot today 😄

Hi all, long time ago, I was using a library that could generate HTTP call functions from clojure.specs. I do not remember the name now though. Is anybody using something like that?

swagger? martian?

ah martian! Thank you 😄

Calling clojure.tools.namespace.repl/refresh in a macro used in CLJS prints an error (ie. :error-while-loading some.namespace) but the rest of the macro continues. No exception is thrown and clojure.core/*e is unbounded. Any idea how to retrieve the error? Right now I throw the exception myself in some.namespace so I know what is going on. I guess that the macro and refresh are not executed on the same thread but I don't know why (since refresh is called from the macro).

*e does not have a binding established when you are compiling your clojurescript, so refresh cannot set! It

@hiredman Oh right, it looks a bit weird but (binding [clojure.core/*e nil] ...) works, thanks!

hi folks. does anyone have any experience with jdbc and hugsql? i am looking at the docs and trying to figure out how to wrap inserts in a transaction. the docs are stated as: https://www.hugsql.org/#using-transactions -- however it does not specify how to actually create a transaction. looking at https://github.com/funcool/clojure.jdbc/blob/master/src/jdbc/transaction.clj was not so enlightening either. maybe someone has an idea? i am trying to concurrently iterate on a collection of data and want to insert transformed data to a sqlite database. sqlite inserts should be wrapped in a transaction for performance reasons, and to avoid errors like "[SQLITE_BUSY] The database file is locked (database is locked)". i am using a manifold stream (like an async channel) to put to a thread that batches inserts into X amount before committing, or after a timeout is reached. i'm using hugsql so would like to take the data in and construct a transaction with the various inserts in and commit after a certain time is reached or a certain amount of inserts is fetched from the stream. any input welcome!

doh, figured it out. with-db-transaction is not a function but a macro, so of course the spec did not make immediate sense. the parameters, [tx db], do not require a tx variable to be created beforehand but instead set a binding within the body, so tx is a transaction created from db and tx takes the place of db in any hugsql calls. lisp syntax is magical but at times confusing without clear idiot-proof commentary.

funcool's clojure.jdbc is also not the library you want to be using

you want https://github.com/clojure/java.jdbc or https://github.com/seancorfield/next-jdbc

funcool's clojure.jdbc has a bit of a cloud over it since some of the early code for the project looks like it could just be copied and pasted from clojure.java.jdbc (in violation of clojure.java.jdbc's license), reading funcool's docs my guess is their position is something like "clojure.jdbc is an implementation of the same api as clojure.java.jdbc so their are bound to be similarities" , but it's not a good look

also they picked a name that used to be a common shorthand used in the community for the library they forked claim to have rewritten from scratch

yes, and rich has indicated clojure.* belongs to him as far as namespaces go

I can link to the Google Groups discussion about the clojure.jdbc author and the whole copying code and stripping all the license and copyright notices...

It still burns that someone would steal my code and pass it off as their own.

But at this point, my main argument about funcool's clojure.jdbc is that it is unmaintained.

HugSQL supports next.jdbc. And next.jdbc's documentation has a getting started guide for HugSQL. So that's the official recommendation.

thanks. my code is old, spent 2020 not really functional 😄 next.jdbc is definitely something i want to pick up. and yes, that was a mis-google, i am indeed using clojure.java.jdbc.

am also a little worried about using ztellman's manifold. seems zach is particularly busy/inactive lately, and noticed that manifold streams try-take! seems to increment :pending-takes continuously even if timeout was reached, not resetting it.

enki.buffers> (def y (stream/stream))
#'enki.buffers/y
enki.buffers> (dotimes [n 16384] (stream/try-take! y 1))
nil
enki.buffers> y
<< stream: {:pending-puts 0, :drained? false, :buffer-size 0, :permanent? false, :type "manifold", :sink? true, :closed? false, :pending-takes 16384, :buffer-capacity 0, :source? true} >>
enki.buffers> (stream/try-take! y 1)
Jan 27, 2021 11:29:29 PM clojure.tools.logging$eval451$fn__455 invoke
WARNING: excessive pending takes (> 16384), closing stream
java.lang.IllegalStateException
	at manifold.stream.default.Stream.take(default.clj:234)
	at enki.buffers$eval38629.invokeStatic(form-init9188607286898067210.clj:17367)
	at enki.buffers$eval38629.invoke(form-init9188607286898067210.clj:17367)
	at clojure.lang.Compiler.eval(Compiler.java:7177)
	at clojure.lang.Compiler.eval(Compiler.java:7132)
	at clojure.core$eval.invokeStatic(core.clj:3214)
	at clojure.core$eval.invoke(core.clj:3210)
	at nrepl.middleware.interruptible_eval$evaluate$fn__29511$fn__29512.invoke(interruptible_eval.clj:87)
	at clojure.lang.AFn.applyToHelper(AFn.java:152)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.core$apply.invokeStatic(core.clj:665)
	at clojure.core$with_bindings_STAR_.invokeStatic(core.clj:1973)
	at clojure.core$with_bindings_STAR_.doInvoke(core.clj:1973)
	at clojure.lang.RestFn.invoke(RestFn.java:425)
	at nrepl.middleware.interruptible_eval$evaluate$fn__29511.invoke(interruptible_eval.clj:87)
	at clojure.main$repl$read_eval_print__9086$fn__9089.invoke(main.clj:437)
	at clojure.main$repl$read_eval_print__9086.invoke(main.clj:437)
	at clojure.main$repl$fn__9095.invoke(main.clj:458)
	at clojure.main$repl.invokeStatic(main.clj:458)
	at clojure.main$repl.doInvoke(main.clj:368)
	at clojure.lang.RestFn.invoke(RestFn.java:1523)
	at nrepl.middleware.interruptible_eval$evaluate.invokeStatic(interruptible_eval.clj:84)
	at nrepl.middleware.interruptible_eval$evaluate.invoke(interruptible_eval.clj:56)
	at nrepl.middleware.interruptible_eval$interruptible_eval$fn__29542$fn__29546.invoke(interruptible_eval.clj:152)
	at clojure.lang.AFn.run(AFn.java:22)
	at nrepl.middleware.session$session_exec$main_loop__29609$fn__29613.invoke(session.clj:202)
	at nrepl.middleware.session$session_exec$main_loop__29609.invoke(session.clj:201)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.base/java.lang.Thread.run(Thread.java:832)

#<SuccessDeferred@11f12ad4: false>
enki.buffers> (def y (stream/stream))
#'enki.buffers/y
enki.buffers> (dotimes [n 16384] (stream/try-take! y 1))
nil
enki.buffers> (stream/put! y 1)
#<Deferred@50a88030: :not-delivered>
enki.buffers> (stream/try-take! y 1)
#<SuccessDeferred@6429f73e: 1>
enki.buffers> y
<< stream: {:pending-puts 0, :drained? false, :buffer-size 0, :permanent? false, :type "manifold", :sink? true, :closed? false, :pending-takes 0, :buffer-capacity 0, :source? true} >>

Manifold has just transferred to the #clj-commons organization and a new maintainer is coming up to speed...

So you could try asking in that channel or open an issue at https://github.com/clj-commons/manifold

Thanks. Yes, will open an issue and may well look at fixing it myself when it's daylight.

I think I just wrote a (as of yet not applied) patch (really a series of patches) for something very like this in core.async.

and it is exactly that, when you have non-deterministic choice, in this case get a value from this stream or timeout, it is not enough to commit to one of the choices, you also need to go back and cleanup the choice not taken

core.async has some code that is run on every put/take/close of a channel that prevents build ups of non-active handlers, but if you are doing operations on channels that don't get a lot of operations (commonly timeout channels, and timeouts have other issues that make this worse) channels can hold on to handlers (and any closed over values) longer then they should

my use case is fetching data to add to sqlite transactions in a separate thread. concurrent inserts fail due to locking, so using a channel to batch inserts is a workaround. however i do not want to specifically wait for N inserts, rather flush on N or after X seconds (to avoid losing data or delays to commits). so a way to either fetch, or time out & commit & loop, is the approach i was going for.

it's been ~1 year or so since i've touched this, or the ecosystem, so i'll have a look around at what is available and how to best approach it. for now, it is midnight, and tomorrow i have work. 🙂 cheers

depending on this that and the other you might be fine with using a java.util.concurrent.LinkedBlockingQueue and passing a timeout to the poll method instead of a manifold stream

mmm, yeah, not actually like the core.async timeouts issue I was thinking, manifold just needs to scan pending puts for realized defers

(and remove them)