@borkdude For some reason your function is failling for some cases, I could only reproduce with that test:
(is (= "(+ 1 1)\n" (#'f.file-management/replace-text "(+ 1 1)\n" "\n" 1 0 1 0)))
It gives a NPE
Not sure it's the only issue thoughI ended up using a very similar when-let*
When generating a pom.xml using tools deps (`clojure -X:deps mvn-pom`), does it retrieve the version from somewhere? There doesn't seem to be something like a :version
option in deps.edn
.
@dromar56 it generates a minimal pom.xml
, it's possible that the mvn-pom
exec function takes various keyword arguments... I don't know that they're documented...
Let me check the source code...
Hmm, I believe this is the function https://github.com/clojure/tools.deps.alpha/blob/f3a2fbb426ae65d5a5c79a756f3b95b463f334a4/src/main/clojure/clojure/tools/cli/api.clj#L88 -- so the only thing it current takes is :argmaps
to help specify how to build the dependencies that appear in the pom.xml
file.
If you are using depstar
to build JAR files, it can update the pom.xml
file for you -- it supports a :version
exec argument, and will update pom.xml
if you say :sync-pom true
.
the version (like most things) should be set in the pom.xml and then deps will sync to it
the underlying pom gen/sync code does actually have the ability to set this, and I expect this to be part of other forthcoming work
Ah thanks, I do use depstar
, I'll switch to that
Got it, thanks. I wasn't sure where was the idiomatic place to put the version, this answers it.
@alexmiller I was actually a bit surprised when I looked at mvn-pom
and it didn't seem to expose the group, artifact, or version that would be generated into the initial pom.xml
...
(so I'm glad to hear that is a future possibility)
I have a function defined in a namespace e.g., user/test-func
I wish to send a keyword :user/test-func
over network and resolve it into the defined function to execute. Is there a way to do it?
I think this only works in clj... I am trying to do it in cljs..
maybe something like:
user> (-> :clojure.string/join
symbol
requiring-resolve)
#'clojure.string/join
Hi, Who can explain this code to me? (:keyword things) What does this code do? I saw this code when invokes
; invoke
(driver.u/database->driver (:database preprocessed)
; defined
(def ^{:arglists '([database-or-id])} database->driver
(memoize/ttl database->driver* :ttl/threshold 1000))
in Metabase source code. Does (:database preprcoessed) is a function invocation or just function parameters?Keywords in Clojure behave like functions when used as the first element of an expression, and cause themselves to be looked up in a map. They aren't 100% the same in all ways, but (:kw foo)
is similar in behavior to (get foo :kw)
Thank you, I got it.
Hi
I need to make println
thread-safe.
I sew an example using *out*
but I was wondering if there is another option?
What I need to do is to make sure my print is thread-safe, while other threads my print in their own way (which I cannot control)
Hello World, Does anyone know an article/video/tutorial on how to work with google sheets? I am task to read/write sheets using clojure. Thank you!
a quick bit of googling turned this up, though it is 5 years old. There is a comment from 2019 saying things were still working https://tech.metail.com/reading-google-sheets-from-clojure/
Yes, robertfw link is great, plus if you would like to understand how could you generate a token and work with that.. good point this gist (I modified for my goals and worked fully .. in the last month) https://gist.github.com/arohner/8d94ee5704b1c0c1b206186525d9f7a7
i use this, but it's still need login each time we run the code, i got stuck on the authentication token & refresh token...
(ns asdf.core
(:require [clojure.edn :as edn]
[clojure.test :refer :all]
[google-apps-clj.google-drive :refer :all :as gdrive]
[google-apps-clj.google-sheets-v4 :refer :all :as gsheets]
[google-apps-clj.credentials :as gauth]))
(defn login
[]
(gsheets/build-service "<http://1080463043693-gidvsert2tugasj4sdtt9odjjv3gledq.apps.googleusercontent.com|1080463043693-gidvsert2tugasj4sdtt9odjjv3gledq.apps.googleusercontent.com>"
"_la38KD6Hc73wPtwEAJZYOfJ"))
;this is just a test to connect
(login)
;here's the function to get the spreadsheet info/cell values
(gsheets/get-spreadsheet-info
(login)
"16qb3Zg4kdD62y7cf0oQ4TrRmq890V19fpnu7hmRpTeg")
(gsheets/get-cell-values
(login)
"16qb3Zg4kdD62y7cf0oQ4TrRmq890V19fpnu7hmRpTeg"
["Sheet1!B9:B11"])
the dependencies are these
[google-apps-clj "0.6.1"]]
@sb how to run that in a project?
I reopen the project and I check it. One moment.
http://cool.no hurries, i'm thankful for that...
I'm using this https://github.com/SparkFund/google-apps-clj and i couldn't get the token & refresh token yet, been learning & working for the gsheet for about 1 week lol... i'm so frustrated here's my failure https://youtu.be/7GAcYNR6LFM
@sfyire I check now the code. I used via lein-env the credentials : api-key and :creds {patht o service json …d49a5.json}. I downloaded from Google Dev console.
and with that I can generate token like “ya27.c…. xYA4Haq”
so no browser interaction
(defn load-creds
"Takes a path to a service account .json credentials file"
[secrets-json-path]
(-> secrets-json-path slurp (json/parse-string keyword)))
(defn create-claim [creds & [{:keys [sub] :as opts}]]
(let [claim (merge {:iss (:client_email creds)
:scope (str/join " " scopes)
:aud "<https://www.googleapis.com/oauth2/v4/token>"
:exp (-> 1 time/hours time/from-now)
:iat (time/now)}
(when sub
;; when using the Admin API, delegating access, :sub may be needed
{:sub sub}))]
(-> claim jwt/jwt (jwt/sign :RS256
(-> creds
:private_key
(#(StringReader. %)) (#(key/pem->private-key % nil))))
(jwt/to-str))))
(defn request-token [creds & [{:keys [sub] :as opts}]]
(let [claim (create-claim creds opts)
resp (http/post "<https://www.googleapis.com/oauth2/v4/token>"
{:form-params {:grant_type "urn:ietf:params:oauth:grant-type:jwt-bearer"
:assertion claim}
:as :json})]
(when (= 200 (-> resp :status))
(-> resp :body :access_token))))
(request-token (load-creds creds))
you get the token and you can use.. the v4 api and v3 api different.. eg
(http/post "<https://analytics.googleapis.com/$discovery/rest?version=v3>"
{:accept :json
:throw-exceptions false
:headers {"Authorization" (str "Bearer " token)}
:form-params form-params
:content-type :json}))
I used for analytics exports, but I think, that is similar to other things
let me digest this... sorry I'm still newbie ahahah
https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets/
(http/get (str "<https://sheets.googleapis.com/v4/spreadsheets/1L....iu3l4?key=>" api-key)
{:accept :json
:headers {"Authorization" (str "Bearer " token)}
:content-type :json})
That is a simple GET request, you can test how to create post request via this documentation.. do you need add valid credentials too etc (allow this request type etc)
I'll try to run the code, and if there's any stuck question arises i'll try to ask... thanks a lot for sharing @sb
Ok, the key the credentials and the scopes. I hope it will work for you!
Thanks, everyone I appreciate all your response and now I ended up using an API key rather than looking for an official client library.:thumbsup:
hi, can someone using windows help, how to escape space character? sadly I couldn't avoid this
(create-claim
(load-creds "/Users/Adrian Imanuel/Google Drive/3. Clojure/Projects/newgsheet/client_secret_1080463043693.json"))
i got an error due to using space
Execution error (NullPointerException) at java.io.StringReader/<init> (StringReader.java:50).
Cannot invoke "String.length()" because "s" is null
type *e
to see the full exception trace, and put that in a snippet here (Ctrl-Shift-Space)
(I'm not sure I agree with the problem analysis around space characters)
(ns newgsheet.core
(:require [cemerick.url :as url]
[cheshire.core :as json]
[clj-jwt.core :as jwt]
[clj-jwt.key :as key]
[clj-time.core :as time]
[clj-http.client :as http]
[clojure.string :as str])
(:import java.io.StringReader))
(defn request-token [creds & [{:keys [sub] :as opts}]]
(let [claim (create-claim creds opts)
resp (http/post "<https://www.googleapis.com/oauth2/v4/token>"
{:form-params {:grant_type "urn:ietf:params:oauth:grant-type:jwt-bearer"
:assertion claim}
:as :json})]
(when (= 200 (-> resp :status))
(-> resp :body :access_token))))
(request-token
(load-creds "c:\\Users\\Adrian Imanuel\\Google Drive\\3. Clojure\\Projects\\newgsheet\\client_secret_1080463043693.json"))
need the stack trace @adrianimanuel
here the stack trace
;; => #error {
:cause "\\Users\\AdrianImanuel\\Google Drive\\3. Clojure\\Projects\\newgsheet\\client_secret_1080463043693.json (The system cannot find the path specified)"
:via
[{:type java.io.FileNotFoundException
:message "\\Users\\AdrianImanuel\\Google Drive\\3. Clojure\\Projects\\newgsheet\\client_secret_1080463043693.json (The system cannot find the path specified)"
:at [java.io.FileInputStream open0 "FileInputStream.java" -2]}]
:trace
[[java.io.FileInputStream open0 "FileInputStream.java" -2]
[java.io.FileInputStream open "FileInputStream.java" 211]
[java.io.FileInputStream <init> "FileInputStream.java" 153]
[<http://clojure.java.io|clojure.java.io>$fn__11496 invokeStatic "io.clj" 229]
[<http://clojure.java.io|clojure.java.io>$fn__11496 invoke "io.clj" 229]
[<http://clojure.java.io|clojure.java.io>$fn__11409$G__11402__11416 invoke "io.clj" 69]
[<http://clojure.java.io|clojure.java.io>$fn__11508 invokeStatic "io.clj" 258]
[<http://clojure.java.io|clojure.java.io>$fn__11508 invoke "io.clj" 254]
[<http://clojure.java.io|clojure.java.io>$fn__11409$G__11402__11416 invoke "io.clj" 69]
[<http://clojure.java.io|clojure.java.io>$fn__11470 invokeStatic "io.clj" 165]
[<http://clojure.java.io|clojure.java.io>$fn__11470 invoke "io.clj" 165]
[<http://clojure.java.io|clojure.java.io>$fn__11422$G__11398__11429 invoke "io.clj" 69]
[<http://clojure.java.io|clojure.java.io>$reader invokeStatic "io.clj" 102]
[<http://clojure.java.io|clojure.java.io>$reader doInvoke "io.clj" 86]
[clojure.lang.RestFn invoke "RestFn.java" 410]
[clojure.lang.AFn applyToHelper "AFn.java" 154]
[clojure.lang.RestFn applyTo "RestFn.java" 132]
[clojure.core$apply invokeStatic "core.clj" 667]
[clojure.core$slurp invokeStatic "core.clj" 6942]
[clojure.core$slurp doInvoke "core.clj" 6942]
[clojure.lang.RestFn invoke "RestFn.java" 410]
[newgsheet.core$load_creds invokeStatic "form-init17689491352809040359.clj" 51]
[newgsheet.core$load_creds invoke "form-init17689491352809040359.clj" 48]
[newgsheet.core$eval7297 invokeStatic "form-init17689491352809040359.clj" 97]
[newgsheet.core$eval7297 invoke "form-init17689491352809040359.clj" 96]
[clojure.lang.Compiler eval "Compiler.java" 7177]
[clojure.lang.Compiler eval "Compiler.java" 7132]
[clojure.core$eval invokeStatic "core.clj" 3214]
[clojure.core$eval invoke "core.clj" 3210]
[nrepl.middleware.interruptible_eval$evaluate$fn__6180$fn__6181 invoke "interruptible_eval.clj" 87]
[clojure.lang.AFn applyToHelper "AFn.java" 152]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.core$apply invokeStatic "core.clj" 665]
[clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973]
[clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973]
[clojure.lang.RestFn invoke "RestFn.java" 425]
[nrepl.middleware.interruptible_eval$evaluate$fn__6180 invoke "interruptible_eval.clj" 87]
[clojure.main$repl$read_eval_print__9086$fn__9089 invoke "main.clj" 437]
[clojure.main$repl$read_eval_print__9086 invoke "main.clj" 437]
[clojure.main$repl$fn__9095 invoke "main.clj" 458]
[clojure.main$repl invokeStatic "main.clj" 458]
[clojure.main$repl doInvoke "main.clj" 368]
[clojure.lang.RestFn invoke "RestFn.java" 1523]
[nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 84]
[nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 56]
[nrepl.middleware.interruptible_eval$interruptible_eval$fn__6211$fn__6215 invoke "interruptible_eval.clj" 152]
[clojure.lang.AFn run "AFn.java" 22]
[nrepl.middleware.session$session_exec$main_loop__6278$fn__6282 invoke "session.clj" 202]
[nrepl.middleware.session$session_exec$main_loop__6278 invoke "session.clj" 201]
[clojure.lang.AFn run "AFn.java" 22]
[java.lang.Thread run "Thread.java" 832]]}
before request-token missing three parts: load-creds, scopes, create-claim fns
;;; Example code for calling Google apis using a service account.
(defn load-creds
"Takes a path to a service account .json credentials file"
[secrets-json-path]
(-> secrets-json-path slurp (json/parse-string keyword)))
;; list of API scopes requested, e.g. <https://developers.google.com/admin-sdk/directory/v1/guides/authorizing>
(def scopes ["<https://www.googleapis.com/auth/admin.directory.user>"
"<https://www.googleapis.com/auth/admin.directory.group>"])
(defn create-claim [creds & [{:keys [sub] :as opts}]]
(let [claim (merge {:iss (:client_email creds)
:scope (str/join " " scopes)
:aud "<https://www.googleapis.com/oauth2/v4/token>"
:exp (-> 1 time/hours time/from-now)
:iat (time/now)}
(when sub
;; when using the Admin API, delegating access, :sub may be needed
{:sub sub}))]
(-> claim jwt/jwt (jwt/sign :RS256 (-> creds :private_key (#(StringReader. %)) (#(key/pem->private-key % nil)))) (jwt/to-str))))
@sb yes, sorry i'm cutting the code, it's my mistake, pls ignore that
ok
ok cool: now that you have the stack trace (in the thread) you can read it like so:
--------top of stack trace ----- opening a file that doesn't exist reading a file http://clojure.java.io/reader related frames clojure.core/slurp newgsheet.core/load-creds (this is the first part of your code that can be recognized) newgsheet.core$eval7297 (something you typed or evaluated in the REPL or editor) clojure/eval clojure.main/repl stuff nREPL crap for editor integration ---------outermost call -------
you absolutely do not need any code to debug this @sb 🙂
call load-creds
again, but make it (prn the-filename)
before it slurp
s the file
You should be able to then call (slurp the-filename)
and see the exact same error without your other code involved
@ghadi reporting that i figured out the missing private_key, it wasn't the code that has problem, but my generated json was the problem. but how did you figure out the missing :private_key? which part of the error stack tells? tbh I still couldn't read all of that
i change this :
(defn load-creds
[secrets-json-path]
(-> secrets-json-path slurp (json/parse-string keyword)))
to
(defn load-creds
[secrets-json-path]
(-> secrets-json-path prn (json/parse-string keyword)))
am i correct?no because prn returns nil
what I want you to do is isolate the problem -- FileNotFound and reproduce it
yes, got the points but didn't get the coding
if you already know what is being passed in, you can do (slurp secrets-json-path)
otherwise
(defn load-creds
[secrets-json-path]
(prn secrets-json-path) ;; add this
(-> secrets-json-path slurp (json/parse-string keyword)))
(load-creds "\\Users\\Adrian Imanuel\\Google Drive\\3. Clojure\\Projects\\newgsheet\\client_secret_1080463043693.json")
"\\Users\\Adrian Imanuel\\Google Drive\\3. Clojure\\Projects\\newgsheet\\client_secret_1080463043693.json"
;; => {:installed {:client_id "<http://1080463043693dtt9odjjv3gledq.apps.googleusercontent.com|1080463043693dtt9odjjv3gledq.apps.googleusercontent.com>", :project_id "clojuresheet-trial", :auth_uri "<https://accounts.google.com/o/oauth2/auth>", :token_uri "<https://oauth2.googleapis.com/token>", :auth_provider_x509_cert_url "<https://www.googleapis.com/oauth2/v1/certs>", :client_secret "_la38KD6ZYOfJ", :redirect_uris ["urn:ietf:wg:oauth:2.0:oob" "<http://localhost>"]}}
this is the result of above codesok now that the string printed, does it look correct? does that file exist? the folder in between "Google Drive" and "Projects" looks suspicious
in my understanding, it's successfully read the path,
the folder called "3. Clojure"
yeah, it looks like it read the file
this comes up a few times but please be aware of the consequences of sharing your client_id and client secret for cloud services
no i already aware of that, i've edited/cut some characters
ok if you call request-token
on the result of load-creds, do you get the same error?
Spoiler alert: all I'm going to suggest is to break the problem down systematically
;; => #error {
:cause "Cannot invoke \"String.length()\" because \"s\" is null"
:via
[{:type java.lang.NullPointerException
:message "Cannot invoke \"String.length()\" because \"s\" is null"
:at [java.io.StringReader <init> "StringReader.java" 50]}]
:trace
[[java.io.StringReader <init> "StringReader.java" 50]
[newgsheet.core$create_claim$fn__7268 invoke "form-init17689491352809040359.clj" 66]
[newgsheet.core$create_claim invokeStatic "form-init17689491352809040359.clj" 66]
[newgsheet.core$create_claim doInvoke "form-init17689491352809040359.clj" 54]
[clojure.lang.RestFn invoke "RestFn.java" 410]
[newgsheet.core$eval7327 invokeStatic "form-init17689491352809040359.clj" 211]
[newgsheet.core$eval7327 invoke "form-init17689491352809040359.clj" 211]
[clojure.lang.Compiler eval "Compiler.java" 7177]
[clojure.lang.Compiler eval "Compiler.java" 7132]
[clojure.core$eval invokeStatic "core.clj" 3214]
[clojure.core$eval invoke "core.clj" 3210]
[nrepl.middleware.interruptible_eval$evaluate$fn__6180$fn__6181 invoke "interruptible_eval.clj" 87]
[clojure.lang.AFn applyToHelper "AFn.java" 152]
[clojure.lang.AFn applyTo "AFn.java" 144]
[clojure.core$apply invokeStatic "core.clj" 665]
[clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973]
[clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973]
[clojure.lang.RestFn invoke "RestFn.java" 425]
[nrepl.middleware.interruptible_eval$evaluate$fn__6180 invoke "interruptible_eval.clj" 87]
[clojure.main$repl$read_eval_print__9086$fn__9089 invoke "main.clj" 437]
[clojure.main$repl$read_eval_print__9086 invoke "main.clj" 437]
[clojure.main$repl$fn__9095 invoke "main.clj" 458]
[clojure.main$repl invokeStatic "main.clj" 458]
[clojure.main$repl doInvoke "main.clj" 368]
[clojure.lang.RestFn invoke "RestFn.java" 1523]
[nrepl.middleware.interruptible_eval$evaluate invokeStatic "interruptible_eval.clj" 84]
[nrepl.middleware.interruptible_eval$evaluate invoke "interruptible_eval.clj" 56]
[nrepl.middleware.interruptible_eval$interruptible_eval$fn__6211$fn__6215 invoke "interruptible_eval.clj" 152]
[clojure.lang.AFn run "AFn.java" 22]
[nrepl.middleware.session$session_exec$main_loop__6278$fn__6282 invoke "session.clj" 202]
[nrepl.middleware.session$session_exec$main_loop__6278 invoke "session.clj" 201]
[clojure.lang.AFn run "AFn.java" 22]
[java.lang.Thread run "Thread.java" 832]]}
oh nook 🙂 so create-claim is barfing
now we need more code @sb 🙂
https://gist.github.com/arohner/8d94ee5704b1c0c1b206186525d9f7a7 @ghadi that was the starter point
I'm sorry not being complete, thought i can pick some function because it has the same error,
load-creds
then create-claim
then request-token
(ns newgsheet.core
(:require [cemerick.url :as url]
[cheshire.core :as json]
[clj-jwt.core :as jwt]
[clj-jwt.key :as key]
[clj-time.core :as time]
[clj-http.client :as http]
[clojure.string :as str])
(:import java.io.StringReader))
(defn load-creds
"Takes a path to a service account .json credentials file"
[secrets-json-path]
(prn secrets-json-path)
(-> secrets-json-path slurp (json/parse-string keyword)))
(defn create-claim [creds & [{:keys [sub] :as opts}]]
(let [claim (merge {:iss (:client_email creds)
:scope (str/join "<https://www.googleapis.com/auth/spreadsheets>")
:aud "<https://www.googleapis.com/oauth2/v4/token>"
:exp (-> 1 time/hours time/from-now)
:iat (time/now)}
(when sub
;; when using the Admin API, delegating access, :sub may be needed
{:sub sub}))]
(-> claim jwt/jwt (jwt/sign :RS256
(-> creds
:private_key
(#(StringReader. %)) (#(key/pem->private-key % nil))))
(jwt/to-str))))
(defn request-token [creds & [{:keys [sub] :as opts}]]
(let [claim (create-claim creds opts)
resp (http/post "<https://www.googleapis.com/oauth2/v4/token>"
{:form-params {:grant_type "urn:ietf:params:oauth:grant-type:jwt-bearer"
:assertion claim}
:as :json})]
(when (= 200 (-> resp :status))
(-> resp :body :access_token))))
I would try to lift out (-> creds :private_key (#(StringReader. %)) (#(key/pem->private-key % nil)))) (jwt/to-str))
into the let
binding
it is very hard to understand
from the trace above, StringReader is saying, "you gave me nil
/ null
to read"
so, is (-> creds :private_key) nil?
(As a style note, don't mix ->
and #(....)
)
the creds
being passed in to create-claim
(coming from load-creds
) doesn't appear to have a :private_key
I gotta run but: break your problems down and debug like a scientist, even if it's slow
alright, i got the point now
will go slow, line by line
thanks a lot for the lesson @ghadi :confusedparrot: going to digest this
given that I have two maps created from frequencies
(so a map of string to the number of that string in the original collection), is there an easy way to subtract the second map from the first, creating a new map that has the subtracted total of the combined keys?
i have a reduce
call that does the trick, but it's nice to use built-ins if they exist
merge-with -
You probably want merge-with @nbtheduke. But I have a qn: how are you sure the keys will be the same? If there are different keys, how do you plan to handle it? If not, then the two maps must have been generated from the same input, is there a way you can generate it only once?
Hi I need to make `println` thread-safe. I sew an example using `*out*` but I was wondering if there is another option? What I need to do is to make sure my print is thread-safe, while other threads my print in their own way (which I cannot control)
println is thread safe
so what behavior are you seeking?
do you mean "not interleaved with other threads printing"?
with-out-str
?
I would like to send a long text to println and make sure the whole text is printed without any other massing the string.
the easiest way is to make the string, then println the full string (don't let println make it)
(println (str/join " " [my args to println]))
instead of (println my args to println)
Got it, thanks a lot!
println will then just print the string to stdout and flush, should not be any interleaving
That's not quite sufficient, since println
prints the string and the newline separately so you can still get some interleaving. I've found that to completely avoid potential interleaving, you need to do (print (str (str/join " " [my args to println]) "\n"))
(it's an edge case so you won't encounter it unless you're printing at a high rate from multiple threads)
Thanks, that is the case I have. This is the service startup where many threads write at the same time in high rate for a few good seconds
That will work well if a key appears in both maps. If it only occurs in the second, I think it will give you the value from the second map. If you want negative of the value from the second map, you'll need a different way.
(it also works well if a key appears in only the first map)
@hiredman and @thomastayac_slack meant to respond sooner but got called away. those are both good answers, but i determined i need to exclude negative numbers, so the reduce
call ended up being the best method
How do you change the in-repl representation of a custom data type? I was hoping there's an interface that provides something like Pythons __repr__
or Clojures Object (toString [_] ,,,)
but that REPLS use when displaying a value?
you can extend print-method
for that (but it's preferred to print as Clojure data)
So that's why newlines get interleaved when println
ing from many threads. Thanks for elucidating
That's a bit of a nasty behavior
Yeah, it surprised me when I first encountered it... 👀