datomic

Ask questions on the official Q&A site at https://ask.datomic.com!
nando 2020-09-27T13:37:35.019900Z

I'm trying to work out how to sort a collection of items nested within the data structure returned from a pull pattern, particularly a pull that uses a reverse lookup. Here's the pull pattern I'm working with:

[:db/id
 {:batch/formula [:db/id :formula/name]}
 :batch/doses
 :batch/date
 {:batch-item/_batch [:db/id
                      {:batch-item/nutrient [:nutrient/name
                                             {:nutrient/category [:category/sort-order]}]}
                      :batch-item/weight
                      :batch-item/complete?]}]
The :batch-item/_batch bit returns a rather large collection and I want to sort it by :category/sort-order and :nutrient/name

souenzzo 2020-09-27T13:39:59.021200Z

@nando you can use #specter with something like (transform [(walker :batch-item/nutrient) :batch-item/nutrient] (partial sort-by :nutrient/category) (d/pull ...))

1👍
nando 2020-09-27T13:44:35.023300Z

So I would wrap the pull in a specter transform? With a query that returns a flat structure, I'd use

(sort-by (juxt :sort-order :nutrient-name)
           (d/q ...

souenzzo 2020-09-27T14:22:20.024Z

#specter will help you to "find some sub-structure and transform it without change anything outside it'

souenzzo 2020-09-27T14:24:02.025Z

once you find what you want to transform (the second argument, know as 'path'. On the example: find a map with this key, and 'enter' this key)

souenzzo 2020-09-27T14:24:54.026Z

in this case the transform function will not by sort-by :nutrient/category, but something like #(sort-by (fn [el] ((juxt ..) el)) %)

souenzzo 2020-09-27T14:26:21.026600Z

the path is [(walker :batch-item/nutrient) :batch-item/nutrient ALL] TLDR; #datomic do not do anything about sorting

nando 2020-09-27T14:50:46.028700Z

Thanks @souenzzo , will look into #specter next.

daniel.spaniel 2020-09-27T18:23:29.029100Z

This is mega important, not just for enterprise, but for any datomic cloud user who wants to preserve their sanity. The sooner the better Stuart. This is huge deal.

daniel.spaniel 2020-09-27T18:25:26.030200Z

Does datomic query syntax allow for group-by? I wanted to group some datums by date and then count them.

nando 2020-09-27T19:00:15.031100Z

I've been looking at the clojure core function group-by for this https://clojuredocs.org/clojure.core/group-by

daniel.spaniel 2020-09-27T19:02:19.031600Z

Has it worked ? I will try it as well .. see what happens .. good idea

nando 2020-09-27T19:06:58.032700Z

I haven't incorporated it into the app I'm working on, but it certainly worked in the REPL

nando 2020-09-27T19:20:42.035700Z

(defn group-nutrients-by-category
  [v]
  (group-by :category-name v))
I've got a datomic query that returns nutrients, and each of these have a category, such as Vitamins, Minerals, Amino Acids, Plant Extracts. I just tried the above, using (group-nutrients-by-category (find-all-nutrients)) and it worked perfectly, as expected.

daniel.spaniel 2020-09-27T19:27:11.036300Z

right, that is doing group-by after the query .. i meant in the query itself ..

nando 2020-09-27T19:30:34.038300Z

It is my understanding that sorting and grouping is done with clojure functions rather than datalog query syntax.

daniel.spaniel 2020-09-27T19:31:51.039500Z

i reckon so. there are some other aggregate function like max, min count, but not group-by or sort-by that are built in

nando 2020-09-27T19:32:39.040200Z

Have you tried to sort the results of a query yet?

daniel.spaniel 2020-09-27T19:32:55.040400Z

oh sure, tis easy

nando 2020-09-27T19:34:20.040800Z

^^^

Joe Lane 2020-09-27T19:35:13.041300Z

Is this not what you mean when you say group-by?

Joe Lane 2020-09-27T19:38:27.042400Z

@dansudol '[:find ?date (count ?e) :where [?e :entity/date ?date]]

Joe Lane 2020-09-27T19:39:06.042900Z

Have you looked at https://docs.datomic.com/cloud/query/query-data-reference.html#aggregate-example

daniel.spaniel 2020-09-27T19:41:23.044600Z

yes, that is pretty close to the query i need Joe, interesting .. i guess if that does the same as group by ( i am reading the examples now ) then that does it .. i am going for something a big more complicated ( count by date range ) but if this works as grouping by date then i am super close to what i want

Joe Lane 2020-09-27T19:44:02.045Z

Are the date ranges contiguous and non-overlapping?

daniel.spaniel 2020-09-27T19:44:14.045200Z

yes

daniel.spaniel 2020-09-27T19:44:54.045900Z

beginning ->end of a month , so finding items whose dates are in that range and counting them up, where let's say the range is a year, so each month, wanted the count of the items ( that have date field on them )

Joe Lane 2020-09-27T19:46:31.046400Z

What datomic system are you using?

daniel.spaniel 2020-09-27T19:46:36.046600Z

cloud

nando 2020-09-27T19:51:23.049Z

If I'm understanding the difference correctly, using group-by will return all records, while using count in an aggregate query will return a single record for each date.

daniel.spaniel 2020-09-27T19:53:44.051100Z

you can't use group-by in the query though, just to operate on the returned data , but the last part is right i reckon

daniel.spaniel 2020-09-27T19:54:41.052200Z

i guess the count by date is kinda grouping dates in a way so there is the element of group by there

Joe Lane 2020-09-27T19:57:55.053700Z

'[:find ?month (count ?e)
  :where
  [(java.time.ZoneId/of "UTC") ?UTC]
  [?e :entity/date ?date]
  [(.toInstant ^Date ?date) ?inst]
  [(.atZone ^Instant ?inst ?UTC) ?inst-in-zone]
  [(.getMonthValue ^ZonedDateTime ?inst-in-zone) ?month]

Joe Lane 2020-09-27T19:58:47.054700Z

Consider the above a sketch, written in slack, untested, likely need to add a few things.

daniel.spaniel 2020-09-27T19:59:20.055600Z

that is pretty hillarious Joe, nifty idea , i will hack around it

Joe Lane 2020-09-27T20:03:04.058700Z

The instant type in datomic is a java.util.Date, so if you want to use the nice .getMonthValue method you'll need some combination of that. There are several other things you could do like make a custom query function to do all the gnarly time conversion stuff in an isolated way. https://docs.datomic.com/cloud/query/query-data-reference.html#deploying Other than that time conversion stuff, this is a pretty trivial query, right? It's basically:

'[:find ?month (count ?e)
  :where
  [?e :entity/date ?date]
  [(my.ions/date->month ?date) ?month]]

Joe Lane 2020-09-27T20:04:21.059300Z

(You might need to use :with ?month in that query, I'd have to think about it...)

daniel.spaniel 2020-09-27T20:04:40.059400Z

pretty much, your idea is good .. me like

daniel.spaniel 2020-09-27T20:26:02.059900Z

interesting @lanejo01 .. this works, very nice ( i made my own database function as you suggested ) slick !

Joe Lane 2020-09-27T20:27:24.060800Z

Great to hear!