Clojurians Log v2

Interesting, hadn't read that one before

I think he makes a great point actually. But I think it would have been nice for him to describe when does BIG become Bloated.

Does this work by design or by accident?

(clojure.core/IllegalArgumentException.)

(`clojure.core/IllegalArgumentException` does not resolve to the class, but the constructor does, it seems)

it's a bug in macroexpand

user=> (macroexpand '(clojure.core/Foo.))
(new Foo)

well I guess it's GIGO

the namespace segment is just ignored (but because of impl details, it must also exist 🙃 )

same as with protocol method impls basically

ish, yeah

I know there are some weird corners in the symbol resolution, was just in that code recently, jira welcome

fixing this would be a breaking change now though

https://clojure.atlassian.net/browse/CLJ-2604

a patch for this is trivial, lmk if you want one

@alexmiller https://clojure.atlassian.net/browse/CLJ-2605

welp :)

oh eh hmm :) I'll close it

done

:gotta_go_fast:

Has anybody ever encountered issues using aliased keywords in .cljc files when both the alias and the alias keyword are inside of reader conditionals for one platform (say :clj) but you're compiling for another platform (say :cljs)? For example

(ns main
  (:require #?(:clj [<http://clojure.java.io|clojure.java.io> :as io])))

#?(:clj (defn io-world [] ::io/world)))

wow, that is strange

I would have expected because the namespaced keyword is restricted to clojure it wouldn't modify cljs compilation.

autoresolution of keyword happens at read time

reader conditionals require all the branches to be readable

Well that's unfortunate. 😕

Same with comment. At least, in this case you can just use a fully qualified keyword.

Oh wow, commented code has to be readable in all branches?

comment is just a macro. there's no mechanism by which it couldn't be readable

I'm just surprised things have to be readable for platforms they don't apply to. I guess I had expected the reader would just skip over sections for other platforms.

yeah its a bit weird but to skip something it has to be read and discarded. to be structurally one form it has to be read. I think the autoresolve of keywords should be inhibited of branches not taken but that's not the case. I think there are some tickets about this kind of problem

https://clojure.atlassian.net/browse/CLJ-2099

I’m making a formdata post request like so:

using http-xhrio in a re-frame handler

{:http-xhrio (http-post "/api/upload-shot-video" {:video body}
   [:upload-success]
   [:upload-error])}
     

where http-post is the following:

(defn http-post [uri params on-success on-failure]
  {:method :post
   :uri (str "<http://d18a6571c2e5.ngrok.io>" uri)
   :params params
   :on-success on-success
   :on-failure on-failure
   :response-format (edn/edn-response-format)
   :format (edn/edn-request-format)
   })

However, I’m getting the following error on the server:

clojure.lang.ExceptionInfo: Malformed application/edn request. {:type :muuntaja/decode, :default-format "application/json", :format "application/edn", :charset "utf-8", :request {:reitit.core/match #reitit.core.Match{:template "/api/upload-shot-video", :data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :result #reitit.ring.Methods{:get #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :get, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :head #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :head, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :post #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :post, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :put #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :put, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :delete #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :delete, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :connect #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :connect, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :options #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :options, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :trace #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :trace, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}, :patch #reitit.ring.Endpoint{:data {:middleware [#function[humboiserver.middleware/wrap-formats] #function[ring.middleware.multipart-params/wrap-multipart-params]], :handler #function[humboiserver.routes.home/upload-shot-video]}, :handler #function[humboiserver.middleware/wrap-formats/fn--9418], :path "/api/upload-shot-video", :method :patch, :middleware [#reitit.middleware.Middleware{:name nil, :wrap #function[humboiserver.middleware/wrap-formats], :spec nil} #reitit.middleware.Middleware{:name nil, :wrap #function[ring.middleware.multipart-params/wrap-multipart-params], :spec nil}]}}, :path-params {}, :path "/api/upload-shot-video"}, :reitit.core/router #object[reitit.core$lookup_router$reify__11769 0x6741233b "reitit.core$lookup_router$reify__11769@6741233b"], :aleph/request-arrived 150075890045367, :aleph/keep-alive? true, :cookies {}, :remote-addr "0:0:0:0:0:0:0:1", :params {}, :flash nil, :headers {"host" "<http://d18a6571c2e5.ngrok.io|d18a6571c2e5.ngrok.io>", "user-agent" "Humboi/1 CFNetwork/1206 Darwin/20.1.0", "content-type" "application/edn", "content-length" "42", "accept" "application/edn", "accept-language" "en-us", "x-forwarded-for" "110.225.238.196", "accept-encoding" "gzip, deflate"}, :server-port 3000, :form-params {}, :session/key nil, :query-params {}, :uri "/api/upload-shot-video", :server-name "localhost", :query-string nil, :path-params {}, :body #object[java.io.ByteArrayInputStream 0x7f2c96a7 "java.io.ByteArrayInputStream@7f2c96a7"], :multipart-params {}, :scheme :http, :request-method :post, :session {}}}
	at muuntaja.core$fail_on_request_decode_exception.invokeStatic(core.clj:132)
	at muuntaja.core$fail_on_request_decode_exception.invoke(core.clj:126)
	at muuntaja.core$create$_decode_request_body__8194.invoke(core.clj:424)
	at muuntaja.core$create$reify__8216.negotiate_and_format_request(core.clj:491)
	at muuntaja.middleware$wrap_format$fn__8285.invoke(middleware.clj:72)
	at humboiserver.middleware$wrap_formats$fn__9418.invoke(middleware.clj:38)
	at reitit.ring$ring_handler$fn__12248.invoke(ring.cljc:326)
	at clojure.lang.AFn.applyToHelper(AFn.java:154)
	at clojure.lang.AFn.applyTo(AFn.java:144)
	at clojure.lang.AFunction$1.doInvoke(AFunction.java:31)
	at clojure.lang.RestFn.invoke(RestFn.java:408)
	at clojure.lang.Var.invoke(Var.java:384)
	at ring.middleware.reload$wrap_reload$fn__4463.invoke(reload.clj:39)
	at selmer.middleware$wrap_error_page$fn__4478.invoke(middleware.clj:18)
	at prone.middleware$wrap_exceptions$fn__4719.invoke(middleware.clj:159)
	at ring.middleware.flash$wrap_flash$fn__8523.invoke(flash.clj:39)
	at ring.middleware.session$wrap_session$fn__8890.invoke(session.clj:108)
	at ring.middleware.keyword_params$wrap_keyword_params$fn__8936.invoke(keyword_params.clj:53)
	at ring.middleware.nested_params$wrap_nested_params$fn__8994.invoke(nested_params.clj:89)
	at ring.middleware.multipart_params$wrap_multipart_params$fn__9126.invoke(multipart_params.clj:171)
	at ring.middleware.params$wrap_params$fn__9150.invoke(params.clj:67)
	at ring.middleware.cookies$wrap_cookies$fn__8841.invoke(cookies.clj:214)
	at ring.middleware.absolute_redirects$wrap_absolute_redirects$fn__9338.invoke(absolute_redirects.clj:47)
	at ring.middleware.resource$wrap_resource_prefer_resources$fn__9186.invoke(resource.clj:25)
	at ring.middleware.content_type$wrap_content_type$fn__9286.invoke(content_type.clj:34)
	at ring.middleware.default_charset$wrap_default_charset$fn__9310.invoke(default_charset.clj:31)
	at ring.middleware.not_modified$wrap_not_modified$fn__9252.invoke(not_modified.clj:61)
	at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
	at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
	at ring.middleware.x_headers$wrap_x_header$fn__8486.invoke(x_headers.clj:22)
	at humboiserver.middleware$wrap_internal_error$fn__9412.invoke(middleware.clj:17)
	at aleph.http.server$handle_request$fn__19409$f__14156__auto____19410.invoke(server.clj:158)
	at clojure.lang.AFn.run(AFn.java:22)
	at io.aleph.dirigiste.Executor$Worker$1.run(Executor.java:62)
	at manifold.executor$thread_factory$reify__14038$f__14039.invoke(executor.clj:44)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.lang.Thread.run(Thread.java:748)
Caused by: clojure.lang.ExceptionInfo: Malformed application/edn in :muuntaja/decode {:type :muuntaja/decode, :format "application/edn"}
	at muuntaja.core$on_exception.invokeStatic(core.clj:284)
	at muuntaja.core$on_exception.invoke(core.clj:267)
	at clojure.core$partial$fn__5839.invoke(core.clj:2626)
	at muuntaja.core$create_coder$decode__8144.invoke(core.clj:322)
	at muuntaja.core$create$_decode_request_body__8194.invoke(core.clj:422)
	... 34 more
Caused by: java.lang.RuntimeException: No reader function for tag object
	at clojure.lang.EdnReader$TaggedReader.readTagged(EdnReader.java:801)
	at clojure.lang.EdnReader$TaggedReader.invoke(EdnReader.java:783)
	at clojure.lang.EdnReader$DispatchReader.invoke(EdnReader.java:549)
	at clojure.lang.EdnReader.readDelimitedList(EdnReader.java:757)
	at clojure.lang.EdnReader$MapReader.invoke(EdnReader.java:680)
	at clojure.lang.EdnReader.read(EdnReader.java:145)
	at clojure.lang.EdnReader.read(EdnReader.java:111)
	at clojure.edn$read.invokeStatic(edn.clj:35)
	at clojure.edn$read.invoke(edn.clj:14)
	at muuntaja.format.edn$decoder$reify__7763.decode(edn.clj:12)
	at muuntaja.core$create_coder$decode__8144.invoke(core.clj:320)
	... 35 more

How to fix this error?

Malformed application/edn No reader function for tag object. You're serializing something that shouldn't be serialized

probably your video body is of the wrong type

I’m trying to upload a video to the server from its url in ios

Here’s more code:

(let [video {:uri (-> coeffects :db :shot-video-uri)}
         body (js/FormData.)]
     (.append body "video" video)
     {:http-xhrio (http-post "/api/upload-shot-video" {:video body}
  [:upload-success]
  [:upload-error]}
    ))

@jtkdvlp body is of type FormData

IIRC FormData has to be the body itself - you cannot wrap it into an extra object like {:video ...}.

Seems like in your case above you can just replace {:video body} with body.

@p-himik unfortunately I’m still getting the error after replacing {:video body} with body

You might want to check the mdn resources on FormData.

Ah, you also have :format (edn/edn-request-format) - don't do that.

yeah maybe you try to send form-data binary array as edn?

@ps Try just removing the line with :format from http-post..

Here is a little JS snippet that might interest you:

const form_entries = (new FormData($form)).entries();
const params = [...form_entries];

// params is a JS-array now

Oops we’re in #clojure here. I thought this is related to #clojurescript

It doesn't have to be an array. You can just send a single FormData object.

The issue is that OP is trying to serialize it as EDN.

@p-himik removing format gives unrecognized request format nil

Yeah, that’s why you’d want to convert it first right?

So this is actually a clojurescript question?

That shouldn't happen because the error is thrown only when :body is nil. Can you post here the full code that you have right now?

You don't want to convert it at all - you just have to send raw FormData because the underlying platform (JavaScript) knows how to handle it. Yes, it ends up being a CLJS question. Or even a cljs-ajax question.

(reg-event-fx
 :upload-shot-video
 (fn [coeffects _]
   (prn "uploading video")
   (let [video {:uri (-> coeffects :db :shot-video-uri)}
         body (js/FormData.)]
     (.append body "video" video)
     {:http-xhrio {:method :post
                   :uri (str "<http://d18a6571c2e5.ngrok.io>" "/api/upload-shot-video")
                   :params body
                   :on-success [:upload-success]
                   :on-failure [:upload-error]
                   :response-format (edn/edn-response-format)
                   ;;:format (edn/edn-request-format)
                   }}
     #_(POST "/api/upload-shot-video"
             {:body body
              :response-format :json
              :keywords? true
            :handler #(dispatch [:upload-success %])
            :error-handler #(dispatch [:upload-error %])})
     )))

Ah, I missed one thing. Replace :params body with :body body.

@p-himik thanks the error has disappeared

but the params are empty in the server

Here’s the handler:

(defn upload-shot-video [req]
(prn "video is! " (-> req :params))
  (r/response {:res "okay!"}))

and I’m getting:

"video is! " {:video ""}

I don't know what you're using on your server so I can't really help. My assumption is that you have the content not in (:params req) but some place else. Maybe (:body req) - at least, that would be logical.

I will probably be a binary stream.

that’s right. there’s a stream at (-> req :body)

Well there you have it. :) Seems like all is fine.

thanks a ton!

NP

@p-himik while we’re at it, do you know how to convert an input stream to a file?

that can be saved to disk?

Look at the functions in the <http://clojure.java.io|clojure.java.io> namespace. I don't exactly remember the way to do it.

Ah right, I assumed that the library used here strictly expects serializable data. Your other answer (changing to :body) explained it for me thanks

How to convert an inputstream to an outputstream?

I try the following:

(http://clojure.java.io/make-output-stream (-> req :body))

but get the following error:

No single method: make_output_stream of interface: http://clojure.java.io.IOFactory found for function: make-output-stream of protocol: IOFactory

@ps I think the most straightforward thing is io/copy to copy the input stream to the ouput

there is no such thing as "converting an inputstream to an outputstream"

@ps actually, thinking more about this - what is the output-stream for?

an output-stream is something you can write to

@noisesmith to convert to a file on disk

how is copying an input stream to it meaningful?

OK

that's different, and yeah, use io/file and io/copy

I have a video as an input stream on the server that I’m trying to load on aws as a file

that's not creating an output-stream to be clear

for that I’m saving it as a file. I thought to save as a file, it first needs to be converted to an outputstream

the file provides an output-stream

you write to that

@noisesmith I have the following now:

(with-open [o (<http://clojure.java.io/output-stream|clojure.java.io/output-stream> "./resources/public/video.mp4")]
  (.write o (-&gt; req :body))
    )

but this gives:

$ scrot -s foo.png
$ clj
Clojure 1.10.1
user=&gt; (require '[<http://clojure.java.io|clojure.java.io> :as io])
nil
user=&gt; (io/copy (io/file "foo.png") (io/file "bar.png"))
nil
user=&gt; 
$ cmp foo.png bar.png
$ echo $?
0

java.lang.IllegalArgumentException: No matching method write found taking 1 args for class http://java.io.BufferedOutputStream

all you need is io/file and io/copy

don't use the .write method

(if you don't understand shell, the above example copies a png file, and cmp verifies that byte for byte the copy is the same as the original)

in the example you are copying a file

I have an input stream

right, the first arg can be an input stream

io/copy is smart enough to do the right thing

so (io/copy (:body req) (io/file "resources/public/video.mp4"))

(doc <http://clojure.java.io/copy|clojure.java.io/copy>)
-------------------------
<http://clojure.java.io/copy|clojure.java.io/copy>
([input output &amp; opts])
  Copies input to output.  Returns nil or throws IOException.
  Input may be an InputStream, Reader, File, byte[], char[], or String.
  Output may be an OutputStream, Writer, or File.

  Options are key/value pairs and may be one of

    :buffer-size  buffer size to use, default is 1024.
    :encoding     encoding to use if converting between
                  byte and char streams.   

  Does not close any streams except those it opens itself 
  (on a File).

also, be cautious about writing to resources/public, usually resources is where you want to put things that would go in a jar, and you can't write a file to it like that

thanks @noisesmith!

unfortunately the video is 0 bytes

it’s supposed to be more than 0 bytes

a problem I've frequently seen is some ring middleware consuming the request body before my code sees it

@noisesmith as far as I know, I’m using two: middleware/wrap-formats wrap-multipart-params

@ps wrap-multiplart-params already wants to put your upload in a file https://github.com/ring-clojure/ring/wiki/File-Uploads

(I have found this annoying in the past, eg. when I'm running on a cloud server, all I'd ever do with that file is write it back to the network for storage elsewhere, but everyone seems to accept that and just read the file back to upload... pointless use of disk)

> By default, uploads are stored in temporary files that are deleted an hour after being uploaded. This is handled by https://github.com/ring-clojure/ring/blob/1.6.3/ring-core/src/ring/middleware/multipart_params/temp_file.clj function.