Clojurians Log v2

here a url where you can see the site in action : https://lit-lowlands-46138.herokuapp.com/

Sorry @roelofw I didn't realize I wasn't in this channel

NP

I like to pull down repos and run them locally but it looks like I can't since I need an api key. First thing I suggest is to update the readme with the purpose, if you can run it locally, design decisions you made, etc.

I can give you the same api key as I use in a pm

@roelofw you can pass a :query-params map instead of building the query-string manually e..g here https://github.com/rwobben/paintings/blob/master/src/clj/paintings2/api_get.clj#L39

@snoe : oke , can I then have the same url as before

you mean the key and format part ?

yeah

oke, I will hit the manual of clj-http how to do that

you can drop everything after the '?' and put it in a map like {:format "json" :key apikey}

oke, but the key must be a secret so I have to pull it out with environ

so it will be (client/get {:as :json :format "json" :key (env :key}) as I understand your right ?

and the sentece above it , will be (str <https://www.rijksmuseum.nl/api/nl/collection/>" id "/tiles?")

not quite (client/get (str "<https://www.rijksmuseum.nl/api/nl/collection/>" id "/tiles") {:as :json :query-params {:format "json" :key (env :key)}}

oke, I can do the same for the other urls , right ?

right, yeah

thanks, I will try it

the rest is pretty good for a first project. I'd look into your editor and find a way to auto-format removing extra spaces and trailing parens

changed it and will try to build it

and it worked

now I will change the rest and update my repo

@snoe : thanks for the remarks , so I learn more and more about clojure

@roelofw you might also want to look at http-kit over clj-http http://stackoverflow.com/a/21473485 depends how many ids you're doing in parallel.

moment, one problem at the time

I'm not suggesting you do, just know it's available

@snoe : can you see if I made a error here :

( str "<https://www.rijksmuseum.nl/api/nl/collection?>")
                 (client/get {:as :json :query-params {:key (env :key) :format "json" :type "schilderijen" :toppieces "True"}}) 

drop the ?

Ì see now a empty output

oke, I changed it and try again

hmm, still empty output

type=schilderij vs {:type "schilderijen"} ?

@snoe : you mean like this : (client/get {:as :json :query-params {:key (env :key) :format "json" "type=schilderijen" :toppieces "True"}})

nope, I see then a error message

no, in github you didn't have the en at the end of the value

you were using schilderij / but you just pasted schilderijen

oke, I did things too fast

I will change it

thanks, that worked well

@snoe , Do you want to see the new code ?

sounds like you got it, but I did notice this last pmap should probably be a map https://github.com/rwobben/paintings/blob/master/src/clj/paintings2/api_get.clj#L54 because merge is so fast

oke, I can change that also

and the maxium api calls that run in parallel are 10

IM looking at http-kit and try to figure out how to change the code t

if the max is 10, i'd stick with clj-http

oke

Thanks for all the remarks

no problem

and here the changed code with your name mentioned : https://github.com/rwobben/paintings

@snoe

https://github.com/rwobben/paintings/blob/master/src/clj/paintings2/api_get.clj#L15 this reduce looks like a map to me using https://clojuredocs.org/clojure.core/subs

@roelofw

you can be right, I takes a map and returns a map

I have to rewrite it then

the only thing I have to figure out is how to deal with 2 variables result and id.

as far as I know map can only take a anymous function with 1 parameter

What happens is that I have a id of this type : NL-SK-C-5 and I need this SK-C-5

so I can do (map (fn [item] (subs id 3) )

@snoe

results is just the intermediate vector you're building up inside the reduce right?

I think you're close with that map

yes, as I see it result is the accumelator

I think also, Im only afraid I loose the rest of the map

how's that? map-id is the list of the NL- prefixed ids and you want to subs each one.

you can do almost anything to a list with reduce, map filter etc are just a level of abstraction above. especially when learning clojure if you ever feel the need to use reduce first look through clojure.core for a function that fits what you want to do more closely

No the map looks like this { id: "NL-SK-C-5" : painter "Rembrandt" :year 1658 }

and I only have to change the id and keep the rest

or do we misunderstood each other

I'm pretty sure read-numbers is returning a vector of ids like ["SK-C-5"]

you are right, Im confusing functions . Too late here to think well

@snoe : I think this one is better (map (fn [item] (subs (:id item) 3) )

I will try it

hmm, that gives a empty seq , even as (map (fn [item] (subs item) 3 ) [] map-id)]

in read-numbers, how are you able to use the :artObjects key when the body is a json string?

I also noticed in the json response that there is a key called "objectNumber" that is exactly what you're trying to get when you substring the id

Because client/get { :as json}) convert the json to a clojure object

that's odd, it's not doing that at the repl

might be because I'm bypassing the middle layer wrappers at the repl

can be map-id looks like this:

("nl-SK-A-2963"
 "nl-SK-C-2"
 "nl-SK-C-149"
 "nl-SK-A-3841"
 "nl-SK-C-1368"
 "nl-SK-A-1718"
 "nl-SK-A-1115"
 "nl-SK-A-799"
 "nl-SK-A-3584"
 "nl-SK-A-2860") 

oke, this one works (map (fn [item] (subs item 3 )) map-id)

@snoe : does this look well or can I improve it

@gdeer81 what output do you get from read-numbers then ?

at the repl you get json. I was pulling out the object numbers with (re-seq #"SK-[A-Z]-[0-9]{0, 4}" body-of-response)

json respons from this

( str "<https://www.rijksmuseum.nl/api/nl/collection>")
                 (client/get {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}})  

that is very wierd

also, I'm running a bare repl with only clj-http as a dep, I'm not running it from the project

side note, why are you calling str when you only have one string?

o, that is a left over from another remark

I think I can delete that

okay, I see where you're concating the id. wow the codebase keeps mutating out from under me lol

yes, that is for me the learning process

learn how I can do things "better"

@gdeer81 this better :

(defn read-numbers
  "Reads the ids of the paintings"
  []
  (let [data (-&gt; (client/get  "<https://www.rijksmuseum.nl/api/nl/collection>" {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}})
                 :body
                 :artObjects
                 )
        map-id (map :id data)
        ids (map (fn [item] (subs item 3 )) map-id) ]
    ids )) 

:as :json is dependent on the cheshire library being present

https://github.com/dakrone/clj-http/blob/master/src/clj_http/client.clj#L125-L129

@hiredman thank you. Okay I spun up my repl with lein try "clj-http" "cheshire" and it gave me enough dependencies to test the function

@roelofw Spoiler alert, you can do all of that with (-&gt;&gt; (client/get "<https://www.rijksmuseum.nl/api/nl/collection>" {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}}) :body :artObjects (map :objectNumber))

you can lose the partial and use ->>

okay, updated the snippet

@gdeer81 oke, and object-number is the number without the NL part ?

and can I make test for testing all functions

I did that is not going to work because of the external api

@gdeer81 yes, this is also working :

(defn read-numbers
  "Reads the ids of the paintings"
  []
  (-&gt;&gt; (client/get  "<https://www.rijksmuseum.nl/api/nl/collection>" {:as :json :query-params {:key (env :key) :format "json" :type "schilderij" :toppieces "True"}})
                 :body
                 :artObjects
                 (map :objectNumber)))  

the code gets shorter and shorter

Time to sleep now

Thanks all

until next time

if you have other remarks , write them down here or make a issue

Have a good day, @gdeer81

will do