Clojurians Log v2

I’ve got a pathom performance problem. I’m hitting a resolver like this:

(defresolver all-products-resolver
  [{:keys [conn ast] :as env} _]
  {::pc/output [{:product/all-products [:product/id
                                        :product/sku
                                        :product/description
                                        :product/precise-price
                                        {:taxable/taxes [:tax/rate :tax/category]}
                                        {:product/tax [:tax/rate :tax/category]}
                                        :product/inventory-tracked?]}]}
  (let [db         (d/db conn)
        company-id (env->effective-company-id env)]
    {:product/all-products (get-all-products db company-id)}))

I was hoping that satisfying the exact query would make pathom just return the result

Perhaps this is the kind of thing you’re restructuring in Pathom 3 😄

A total shot in the dark, but would it help just to specify :product/all-products in the output, like ::pc/output [:product/all-products]? We have a query that has an output that is a map and we don't specify any of the sub-keys. We only query for the top-level key.

@tony.kay there is a escape feature for this situation, add the metadata ::p/final true to the list (when returning from the resolver), by doing so Pathom will skip processing it completly

the reason is pathom don't know its exact data, so it has to scan anyway, doing a check for each data shape could have a similar cost, with the user declaration Pathom accepts that the user know more about it, and just skips processing

@tvaughan better to always keep the nested on the output, it doesn't affect running in any way, but improves the intelisense of pathom viz for queries (better auto-complete)

Good to know. Thanks @wilkerlucio

thanks, I had a vague memory there was an escape hatch, but could not remember

no go. Does it matter what order the readers are installed?

I added debug breakpts in pathom, and only saw lots of calls to nested resolution..never saw that part of the program ever get the top-level list…

Map reader’s where I found that ::p/final, and when I watch what it does, I only ever see the individual nested entities. It never sees the list, so never skips it

Yeah, I’m getting into here in connect:

(defn- process-simple-reader-response [{:keys [query] :as env} response]
  (let [key (-> env :ast :key)
        x   (get response key)]
    (cond
      (and query (sequential? x))
      (->> (mapv atom x) (p/join-seq env))

      (nil? x)
      (if (contains? response key)
        nil
        ::p/continue)

      :else
      (p/join (atom x) env))))

key is :products/all-products, and x is my list of 10,000 things, but it ignore metadata here and just processes them all anyway…unless, the query is nil

the thing that takes care of it is the map-reader

https://github.com/wilkerlucio/pathom/blob/214834442615352df15c854e44d537bf2ced96a2/src/com/wsscode/pathom/core.cljc#L758

I’m tracing code with breakpoints

that is where I land, and then it does all 10000

I can't find process-simple-reader-response in pathom, isn't this in your codebase?

connect.cljc

v 2.2.30

ah, right

did you tried bumping to 2.3.0? I think that fn got removed

no, but 2.3.0 was waaaaay slower

can’t risk that in prod

I guess I could try and diagnose that, but I’ve already spent 5 hours on what should have been easy optimizations 😄