Clojurians Log v2
reveal
does reveal makes use of datafy/nav protocols?
Morning! (at least it’s morning here in Sweden)
A lot to unpack here, but I’m glad you resolved your issue @wilkerlucio!
I can describe how datafy/nav works in Reveal :) When maps are printed to Reveal, map keys are annotated with corresponding values, and values are annotated with corresponding keys, and both keys and values are annotated with collection they are in. That provides all necessary information for actions to do nav (coll, k, v) when some key OR value is selected: the counterpart will be there. The relevant code is here: https://github.com/vlaaad/reveal/blob/master/src/vlaaad/reveal/stream.clj#L202-L218 Reveal also performs datafy-then-nav when it suggests nav action, but there is a twist: datafied data structure might be substantially different from the original one. For example, entity map from datomic might have only preloaded keys in its default form, but perform a request to load all its keys when datafied, so we can then lookup related values from those keys:
;; original entity:
{:db/id 123}
;; datafied entity:
{:db/id 123
:user/name "fred"
:user/orders "not loaded, nav this key to load"}:user/orders key because it’s not in the map — it appears when datafication happens. That’s why Reveal’s nav action does not suggest itself if datafication is different: you are missing out on other keys. Instead, you should select the coll and execute datafy action to get those keys, and only then nav in the returned data structure. This is why Reveal suggests datafy action only when datafication is different, by the way — if it’s the same, you can just nav in the printed data structure.
Great explanation! Thanks, @vlaaad!
I see a nav option in the menu, but that doens't seem to be using the nav protocol
Yes, it does. I use it quite a bit with next.jdbc results.
Unlike REBL, it doesn't datafy things automatically, but when you choose nav, it calls datafy and then nav.
(because you can only nav things that have been datafy'd)
@wilkerlucio Does that help?
kind of, but I think I'm finding a bug in reveal
I'm looking at the nav action definition
(defaction ::nav [x {:vlaaad.reveal.nav/keys [coll key val]
:or {key ::not-found
val ::not-found}}]
(let [datafied-coll (d/datafy coll)]
(when (= datafied-coll coll)
(cond
(not= key ::not-found) #(d/nav datafied-coll key x)
(not= val ::not-found) #(d/nav datafied-coll x val)))))and after some debug, I notice the :vlaaad.reveal.nav/key comes with numbers for vector items, but comes blank for map entries
trying to understand, maybe its because Im using a custom map type
still digging on it
That does look a bit weird... why would the datafied coll be equal to the original coll?
I guess to make sure its datafied
that check is to define if the action itself must be presented or absent
That makes no sense. datafy can produce something completely different from the underlying data.
but the datafy of datafy should always be the same value, right?
Er, no. Are you saying it's calling datafy twice?
I believe the intention of this check is to only offer "nav" when Reveal thinks the data is datafied, and one way to check that is calling datafy on it again, I guess the property is true: (defn datafied? [x] (= x (datafy x)))
but I start doubting my logic now
yeah, maybe the check should be about the parent implementing Datafiable
Lots of things can implement Datafiable
It looks like @vlaaad has changed this at some point and now datafy is offered as a separate action, at least sometimes.
(find-ns 'user) produces a value that can be datafy'd for example
I'm trying to create my own custom action
but I'm getting confused with the meta-data provided by Reveal, it seems incomplete for properly call nav
Yeah, try this:
(with-meta {} {`p/datafy (fn [x] (with-meta {:a 1 :b 2} {`p/nav (fn [coll k v] (if (= :b k) (* 2 v) v))}))})Then right-click > datafy it. Then go to :a in the result and right-click > nav -- you'll get 1, then go to :b in the result and right-click > nav and you'll get 4.
So it works, but the code in the nav action is pretty counter-intuitive.
I just got it working
the problem is related to lazy navigation
with the current nav impl it doesn't properly use the nav protocol to find lazy data, as far as I understood
I got it to work with this action:
(rx/defaction ::my-nav
[x {:vlaaad.reveal.nav/keys [coll key val]
:as m
:or {key ::not-found
val ::not-found}}]
(when (satisfies? clojure.core.protocols/Navigable coll)
(let [source (-> coll meta :clojure.datafy/obj)]
(cond
(not= ::not-found val)
#(d/nav source x val)
(not= ::not-found key)
#(d/nav source key (get coll key))))))its similar, but note in my version I use the nav in the source object (which comes as part of the datafied data meta)
That's not right.
this way, the nav can return data that is different from what datafy returned
allowing for lazy navigation
You're trying to nav on the original object which is wrong.
You should nav on the result of datafy.
let me try to bring my situation, maybe it will make more sense
In a lot of real world cases, datafy is what adds the nav protocol implementation.
You can datafy some arbitrary thing and the result will be nav'able.
the problem is that, how do I datafy without doign recursive explosion?
in my case, I have a DS that's a map like, but it has some cached data, and some unrealized data
if I realize everything on datafy, it causes stack overflow before it goes infinite recursion
datafy is a no-op on most things but you must call datafy first before nav
so, I need to return something to say that those keys exist, but I can't evaluate then ahead of time
maybe I'm using it wrong, but this is the only way I found to be able to know the navigatable keys, but only realize on nav
You are doing it wrong 🙂
is there a way to solve my problem "doing it right"?
Explain your problem again...
I have this data structure that contains cached data, and some keys that are "possibly available"
like a datomic entity, where a read may or may not cause an index download, but in my case I have an arbritary number of keys, that may return other complex data in it
So datafy should produce a pure data representation that has those keys but doesn't have to realize them... so they're placeholders that can be displayed as pure data.
my goal is to make a navigatable environment on this special map, so during datafy I need to expose all possible keys, but can't evaluate all of them (only the cached, that I know its a limited space, the "possible" is bound to infinity)
yeah, that's what I'm doing
I return a special keyword on those cases
ah, I think I may realized the solution, use metadata on the values instead of the protocol on the parent type
Have a look at how next.jdbc does it
It does lazy navigation through a database based on assumed foreign keys.
datafy adds the nav protocol implementation that does the DB fetch when invoked.
If you're in doubt, fire up REBL and try it in that: it definitely does datafy/nav "the right way" so if your impl doesn't work in REBL, it's not right.
Reveal is... a bit odd in that respect...
for some reason REBL is crashing my process when I try to get it up here
ok, I got the nav to work as expected now, thanks for pointing the direction @seancorfield 🙏
Excellent!
is there a way to make the maps print in reveal with the keys sorted?
I was considering doing that, but was worried it might give users an incorrect view of their program: they might get an impression all maps are always sorted. I usually use eval-on-selection and write sort when I want something sorted
I think would be nice to have the option, because for example if I try to to do this with a Nav object, then I can't navigated on the sorted list, and I quite like the view of regular map on sorted. by personal experience with Fulcro Inspect I know that I'm mostly happier having then sorted, would you consider having this as a configurable thing?
you can have custom :vlaaad.reveal.stream/type meta key for your data structure and make that sort the keys at the point of printing
I think it would be better to have it for everything, I guess its a preference on the user side, in my case I normally deal with quite large maps, unsorted keys is really painful with those
could be a pref like the Theme on Reveal, but I would like to apply to every map by default
Putting it into pref is a good idea, thanks! I’ll give it a thought
No, but you can right-click > table view and then sort keys or values
or Space, type sort, Enter
(my :dev alias in my dot-clojure repo has an auto-table-display function built-in)
another thing on maps, had you consider the option to collapse map values (and sequences)?
@wilkerlucio I released 1.1.178 that bumped text max length in tables a little. Ideally that could be done as a preference, but it’s a bigger task
@genekim My pleasure 😊
Unfortunately, not truncating the keys is not possible today, since keys can be so long (e.g. (range 1000000)) that trying to show them as JavaFX text nodes will kill UI performance. Maybe one day I’ll be able to implement some sort of intelligent text nodes that render themselves on-demand, but it’s a big task-small payoff, so maybe later ¯\(ツ)/¯
I thought about that. There is no decision yet, but table view is semantically pretty close to collapsing values to get overview of keys/items, and structural navigation allows fast jumping between values in the output panel, so that's pretty low priority for me, but I keep it in mind/todo list
one bug I see on the table view is that the keys column content doesn't resize when you resize the column
and since I use quite large keys that's also a problem
Hmm, can you show how it looks?
not right now, but I can send later
in summary, it keeps "..." at the end of the keys
and when I expand the column size, they remain as "..."
Ah, now I understand! I probably should bump the max length of printed object
Hey there! I just released 1.1.177 that is almost ready to be 1.2.* release with the UI improvements in the area of result navigation:
• now you can have multiple result panels by pressing Ctrl+Enter in the action popup — this will make comparisons between different results/views easier;
• switching between tabs is more discoverable (there are buttons with tooltips);
• in every result panel you can see a result tree that shows all items in this panel hierarchically (available with Ctrl+Up);
I'll fix some minor issues and then will announce it as 1.2.*
Very nice! Just in time for my presentation tonight, where I'll be showing off my workflow which includes Reveal!