Clojurians Log v2

So i'm having trasient issues with load-field-action when it's used twice back-to-back

transient*

if I load-field-action on field :x, and :y, :y wins because it comes later

and :x is given :untangled.client.impl.om-plumbing/not-found

loading field :x works fine. alone. loading :y works fine alone. when it's back to back, the one that comes last wins

here's my current backend code that has the bug

(defmethod api-read :entitlements
  [{:keys [parser query] :as env} key params]
  (parser env query))

(defmethod api-read :ent-profile
  [env key {:keys [sso-id]}]
  (auth/profile-by-id! sso-id))

(defmethod api-read :ent-details
  [env key {:keys [sso-id]}]
  (auth/entitlements-by-id! sso-id))

it's a recursive parser

here's LOG output

16-08-17 23:49:31 fabric INFO [admin.system:21] - Read Request: [ :entitlements [(:ent-details {:sso-id "54d2497fe3e8d00800893017"})]
 ] 
16-08-17 23:49:31 fabric INFO [admin.system:21] - Recursive Read Request: [ :ent-details  ] ({:sso-id "54d2497fe3e8d00800893017"})
16-08-17 23:49:45 fabric INFO [admin.system:21] - Read Request: [ :entitlements [(:ent-profile {:sso-id "54d2497fe3e8d00800893017"})]
 ] 
16-08-17 23:49:45 fabric INFO [admin.system:21] - Recursive Read Request: [ :ent-profile  ] ({:sso-id "54d2497fe3e8d00800893017"})

so you can see we get 2 read requests

in this case :ent-details came first, so it gets assigned :untangled.client.impl.om-plumbing/not-found in the app-state

#_ (do

     (transact! [(entitlements/load-profile {:sso-id "54d2497fe3e8d00800893017"})])
     (transact! [(entitlements/load-entitlements-details {:sso-id "54d2497fe3e8d00800893017"})])
     )

if however at the REPL, I evaluate the first transact!, and wait, then evaluate the second transact!

then the bug doesn't reproduce (both fields get loaded data)

finally if I change my api-read code to the following (always return both fields, even though only 1 is requested at any given time) , then the bug goes away

(defmethod api-read :entitlements
  [{:keys [parser query ast] :as env} key params]
  (let [sso-id (-> ast :children first :params :sso-id)]
    {:ent-profile (auth/profile-by-id! sso-id)
     :ent-details (auth/entitlements-by-id! sso-id)}))

https://github.com/untangled-web/untangled-client/issues/25

@tony.kay ^^^

Hey there

Yeah, I saw the issue. Not had time to look at it

I've never tried it myself, but I did think it would work. On second thought, I'm considering what happens in the pipeline. When you add two things at the same time it tries to combine them into a single query to process all at once. That means that the queries will run in order, and since they are keyed to the same thing (the result is a map, and the key is the ident) that this bad thing will happen.

@jasonjckn There are a few possible "right things" we could do. One is say you should write the query yourself (that's all load-field is really doing: using load-data, but writing the query for you). Another would be to scan the queue and "merge" (or at least warn) of dupe keys. The easiest is probably to add support in load-field to take a single keyword or a vector of keywords. The latter would just generate the proper query. Then it would be considered incorrect to call it twice at once.

We could also remove the "merge" bit from the queue processing. That was intended to make things more efficient, so I'm not really wanting to do that. I mean, load-data can exhibit a similar problem and it sort of breaks composition. So in that sense it is a bit of a concern

So i do like accepting a vector of keywords in load-field

definitely want that regardless

middle ground would just be to make the merge smarter. Leave the things that have dupe top-level keys as separate network requests. This restores composability

two load-field would run two network requests separately (unless you told it two fields with one call)

Yah, it sounds like the merge logic is broken, there should be no real difference between transact! twice back-to-back and transact! wait 1 second transact! but this merge 'optimization' is making these cases behave widly differently, so i would make sense to fix the merge or remove it until it works

right, i like that "two load-field would run two network requests separately (unless you told it two fields with one call)"

Well, the idea is that initial loads might ask for 10 different things, and you'd like them to go all at once...but now that we have :parallel it is less of a concern

i'm all for the optimized version, but until it works proper, may as well just have 2 network requests

although parallel uses separate network requests, it just doesn't serialize them

nods

Yeah, I think the general fix is to the merge

any idea when we can get this fixed?

I can take a look this afternoon I think

great, let me know if there's anything I can do to help

well, you can fix it 😉

yah lol

adding support for vector to load-field is probably relatively easy

well, it may take me a while to get up to speed , yah the latter is more suitable for me

feel free to skip the latter

perhaps i'll do it one day / it's not as pressing

although it's probably just 1 line change

actually load-field is not an easy fix

would be kind of invasive

@jasonjckn (or anyone wanting to get their hands dirty) I added TODO markers in u.c.impl.data-fetch/mark-loading function. If you do those, it should fix the load-field issue. It is basic stuff. Note that the items in the items-to-load sequence have data access helper functions (e.g. data-query returns the query for the item to load).

this is on develop on untangled-client

Volunteers let me know. I will work on it later if no one picks it up.

Nice, i will take a look

let's say you do the optimized version where if the Q contains 2 items with the same key, you merge the subquery, is it really that hard to get right? consider my parser

(defmethod api-read :entitlements
  [{:keys [parser query] :as env} key params]
  {:value (parser env query)})

(defmethod api-read :ent-profile
  [env key {:keys [sso-id]}]
  {:value (auth/profile-by-id! sso-id)})

(defmethod api-read :ent-details
  [env key {:keys [sso-id]}]
  {:value (auth/entitlements-by-id! sso-id)})

16-08-17 23:49:31 fabric INFO [admin.system:21] - Read Request: [ :entitlements [(:ent-details {:sso-id "54d2497fe3e8d00800893017"})]] 
16-08-17 23:49:31 fabric INFO [admin.system:21] - Recursive Read Request: [ :ent-details  ] ({:sso-id "54d2497fe3e8d00800893017"})
16-08-17 23:49:45 fabric INFO [admin.system:21] - Read Request: [ :entitlements [(:ent-profile {:sso-id "54d2497fe3e8d00800893017"})]] 
16-08-17 23:49:45 fabric INFO [admin.system:21] - Recursive Read Request: [ :ent-profile  ] ({:sso-id "54d2497fe3e8d00800893017"})

16-08-17 23:49:31 fabric INFO [admin.system:21] - Read Request: [ :entitlements [(:ent-details {:sso-id "54d2497fe3e8d00800893017"})
                                                                              (:ent-profile {:sso-id "54d2497fe3e8d00800893017”})]] 
16-08-17 23:49:31 fabric INFO [admin.system:21] - Recursive Read Request: [ :ent-details  ] ({:sso-id "54d2497fe3e8d00800893017"})
16-08-17 23:49:45 fabric INFO [admin.system:21] - Recursive Read Request: [ :ent-profile  ] ({:sso-id "54d2497fe3e8d00800893017”})

@currentoor @therabidbanana did you guys roll your own charts or did you use a library? trying to figure out how best to do charts with untangled and am curious to hear your experience

We're using c3

It's not too bad @kenbier did much of the heavy lifting there