Clojurians Log v2

Hello! I have a uni project to do a distributed app service (e.g. a distributed lock server) - with emphasis on the service A friend of mine approached me to perhaps take this chance to learn Clojure - but I'm not sure how to evaluate the feasibility of doing so in Clojure vs say Go. Not sure if anyone is able to offer any advice before I look into it further 😮 The requirements are as follows: - Maintain correctness of the application w/o faults - Should provide scalable service (#nodes supported) - Fault tolerance - Byzantine fault tolerance (Bonus) - Implement a suitable consistency model

There's way more literature and implementations about all of that on the Internet in Go for sure, and IMO, this in area where Go is used the most, low-level distributed network services

So depends more on if you value community/ecosystem resources.

There's lots resources for Java too I guess, the Clojure world just reuses the Java implementations.

Thanks @kaxaw75836, I think it was too tough to find more members who would do clojure anyway. Maybe something to explore, but not within my academics i guess 🙂

I’m trying to upload a video to the server as follows:

(reg-event-fx
 :shoot-video
 (fn [coeffects [_ !camera]]
   (if (-> coeffects :db :shoot-video)
     (do
       (prn "stopping shoot video")
       (. @!camera stopRecording)
       )
     (go
       (prn "starting shoot!")
       (let [uri (.-uri (<p! (. @!camera recordAsync)))]
         (. MediaLibrary saveToLibraryAsync uri)
         (prn "dispatching with this uri" uri)
         (dispatch [:save-uri uri])
         (dispatch [:upload-shot-video]))))
   {:db (assoc (:db coeffects) :shoot-video (not (:shoot-video (:db coeffects))))}))

(reg-event-fx
 :upload-shot-video-server
 (fn [coeffects [_ blob]]
   (let [body (js/FormData.)]
     (.append body "video" {:name "movie.mov" :type "video/quicktime" :uri blob}  "video.mov")
     (.append body "key" "VAL")
     {:http-xhrio {:method :post
                   :uri (str server-uri "/api/upload-shot-video")
                   :body body
                   :on-success [:upload-success]
                   :on-failure [:upload-error]
                   :format (json-request-format)
                   :response-format (raw-response-format) #_(edn/edn-response-format)}}))
 

(defn upload-shot-video [req]
  (prn "uploading video")
  (prn "video is! " (-> req :multipart-params))
  (prn "video is " (-> req :params))
  (prn "video before is " (slurp (-> req :body)))
  (.reset (-> req :body))
  (prn "req full" (-> req))
  (prn "video after is " (-> req :body))
  (prn "video is! " (-> req :multipart-params))
  (<http://clojure.java.io/copy|clojure.java.io/copy> (-&gt; req :body) (<http://clojure.java.io/file|clojure.java.io/file> "./resources/public/video.mov"))

However, the request is not a multipart request, so I don’t see any entries in :multipart-params key

and the entries I do see in the params key doesn’t have a :tempfile key so I can’t access the file that’s uploaded

Instead I see

{"_parts":[["video",{"_data":{"size":2971246,"blobId":"D002459C-47C5-4403-ABC6-A2DE6A46230A","offset":0,"type":"video/quicktime","name":"DCDE604A-954F-4B49-A1F9-1BCC2C2F37BC.mov","__collector":null}}],["key","VAL"]]}

there’s a filename in the request map. Where is this file stored?

I’m unable to extract the data from the request since there’s no tempfile key

and the request is not multipart

You can't pass a clojure map like that to a function that expects a js object

I am no js dev but these docs for append on form data https://developer.mozilla.org/en-US/docs/Web/API/FormData/append don't seem to match your 3 arg call to that method

https://developer.mozilla.org/en-US/docs/Web/API/FormData/Using_FormData_Objects has a js example, using form data to do a multiparty file upload, and you can see they construct a blob object, with the mimetype part of that, and pass it directly to append

@hiredman, after changing to this:

(reg-event-fx
 :upload-shot-video-server
 (fn [coeffects [_ blob]]
   (let [body (js/FormData.)]
     (prn "blob is " blob)
     (.append body "video" blob)
     (.append body "key" "VAL")
     {:http-xhrio {:method :post
                   :uri (str server-uri "/api/upload-shot-video")
                   :body body
                   :on-success [:upload-success]
                   :on-failure [:upload-error]
                   :format (json-request-format)
                   :response-format (raw-response-format) #_(edn/edn-response-format)}}))
)

You should ask in #clojurescript, but I would start by reading some docs. I don't know what library you are using that is actually making the request, but docs on that, I also think setting the format to (json-request-format) seems suspicious

So check the docs on that

@hiredman I’m using the cljs-ajax library

The readme for https://github.com/JulianBirch/cljs-ajax says it doesn't support file uploads

Oh, I misread it

@hiredman it says it doesn’t have “doesn’t have any other support for file uploads”

https://github.com/JulianBirch/cljs-ajax/issues/122 has some examples

@hiredman I’m using :body and not :params as mentioned

wrapping the map with (clj->js) worked.

@hiredman Thanks a lot. You are a savior. Had been stuck on this for two whole days.

How do we test future in clojure?

I want to test if a function which is passed to to future has been called or not

(def state (atom nil))
(defn foo [] (reset! state true))
(def fut (future (foo)))
@fut
(is @state)

or if you want to test the return value of foo, just return it from the future

and then test the return value of @fut

Thank you for the quick reply 🙂

@borkdude Is there a way to do it without atom? Or just checking if the future is initiated?

it depends on your usage. your question was: test if a function has been called, so I assume you want to test some side effect. the side effect depends on your specific program

Yes, right

you can do the same with promise as well

but it's the same really

you can also use a lib like spy

https://github.com/alexanderjamesking/spy

I quite like just relying on promise or atom (if you need to mutate more than once) personally

Just checking, if I get the current eco system correctly. I'm a beginner, so bear with me, if what I type sounds strange to you. • Clojure is a hosted language on the JVM, which makes it easy to use Java libs with interop. • There are implementations with JS and .Net, does that mean I can run .Net and JS code like the Java Interop? • There is clj-python, I understand, that it runs python code on the side and reimports the results back into the clojure environment. So I can call (slower than normal?) Python libs like Java interop? • With GraalVM that is hosting Python on the same VM as my Clojure, does that just speed up my Python calls because they don't need to be external? What is the benefit to the clj-python lib?

clj-python can utilize an entire python ecosystem. GraalVM provides a Python 3.8 compliant runtime.

yes on the first bullet. On the second bullet, ClojureScript is compiled to JavaScript, and has ways to call JavaScript libraries from ClojureScript code, which syntactically are often similar to calling Java methods from Clojure. I haven't used ClojureCLR, but I believe it can make calls to .Net libraries via interop similarly to Clojure->Java calls.

I wouldn't risk anything important on the continued future of the .Net version right now. It seems to have lost momentum, even at Microsoft :-)

I know of one library for generating clojure documentation from tests - http://helpshift.github.io/hydrox/ are there any others? Preferably one that doesn't depend on leiningen

(there’s also #babashka which is clojure as a scripting language (basically), and it has a variety of interop options)

would this be "too clever", shortcutting cond's expansion?

(defmacro cond*
  [& clauses]
  (when clauses
    (let [fst (first clauses)]
      (if (keyword? fst) (second clauses)
          (list 'if fst
                (if (next clauses)
                  (second clauses)
                  (throw (IllegalArgumentException.
                          "cond requires an even number of forms")))
                (cons `cond* (next (next clauses))))))))


(require '[clojure.walk :as walk])

(walk/macroexpand-all '(cond (odd? x) x :else 1))
;;=> (if (odd? x) x (if :else 1 nil))

(walk/macroexpand-all '(cond* (odd? x) x :else 1))
;;=> (if (odd? x) x 1)

seems reasonable, but at least for the jvm and js, wouldn't the JIT also do this kind of analysis?

probably, but why rely on that if you can cut it out earlier

I would have expected Compiler.java to have done a similar trick, but can't find it

trying to think of a case where evaluating a keyword may have some sort of side effect that a program might intentionally or unintentionally depend on, but I can't think of any

any compile-time constant object could be treated as an else there

Interested in the answer

sure, keyword? could be (constant? ...)

keyword certainly is common, but could also be string, true , 42 , etc

I mean the presumption here is that it's faster but probably doesn't matter with jit

it's sort of contrived, but a deepwalking macro or code analysis tool that uses clojure.walk/macroexpand looking for keywords would be affected by this.

it would be faster in an interpreter, less code to process during evaluation. just wondering of what could go wrong

so if you had a macro that would automatically produce defs for all keywords in a macro-expanded expression, it would then fail to find any keyword that is a test in a cond expression

maybe if a macro that was called later on had a side effect

but maybe one should not rely on that

as well as any code analysis tool looking for keywords that macro expanded first

so it may introduce some edge cases

small paper testing this hypothesis: > Small, deterministic programs reach a steady state of peak performance. https://arxiv.org/abs/1602.00602

like:

(cond (odd? x) x :else 1 :foobar (macro-that-launches-rocket))

Rather than changing cond , is it feasible for the interpreter you have in mind to notice (if <constant-expression> ...) forms and optimize those 'in place'?

yeah, that's also possible

That would cover this case, and potentially others.

right

cool paper about the reliability of the JIT and how long (if ever) it might take to warm up

@borkdude @alexmiller a ticket+patch exists somewhere for if’s with constant tests

➕ why would you want to do this on a specific macro instead of a simple general optimisation for if in the compiler

@bronsa yes, I also wondered about the compiler, I didn't see specific code for that

thanks

the clojurescript compiler does a lot of similar kind of, I dunno what to call them, peephole optimizations? I can't think of how this would break anything, but I am wary of that kind of thing in general because as a result cljs has more corner cases with core.async, and sometimes you can get weird double code evaluations

(I guess it isn't the compiler that does it, it is the macros themselves)

https://clojure.atlassian.net/browse/ASYNC-91 and https://clojure.atlassian.net/browse/ASYNC-128 are examples of cljs+core.async issues due to macros trying to be too clever

wow, interesting

> also or and and in clojurescript both run analysis as part of the macro expansion, which can lead to macros be expanded multiple times yeah, I can see that leading to unexpected behavior...

there is another issue somewhere, the cljs compiler I think tries(tried?) to optimize literal list construction in a way that caused multiple evaluations as well

I've also seen some similar things with side effects in CLJS spec macros