Clojurians Log v2

I came to the weird realization that GraphQL does not collect a graph of data. It is only collecting a tree. It is not a query language to gather graphs.

I thought trees were technically graphs. even though it doesn't collect a graph, doesn't it search a graph?

By itself, it doesn't search anything - it's just a query language. How the actual implementation is done is, well, an implementation detail.

GraphQL was made for decorating html trees, that’s why it returns a tree.

Do you have a source that confirms the first part of that sentence? Because that by itself sounds unlikely.

well .. graphs can be implemented as trees, but I recently met a situation where I really need to represent a graph of values, and GraphQL and EQL can’t do that.

Hold on. You need to represent a graph. But you're looking for a query language. I don't see how the two are connected.

you can represent a graph of values, but some of the nodes would need to represent references to other parts of the tree

I have no source, just my impression that GraphQL was made for improving data loading at the time where everybody was doing React.

@p-himik I encountered a situation where I need to express a query in the shape of a graph instead of a tree.

This is a graph with a single cycle represented as an EDN structure with an ID convention on top of it:

[{:id 1, :sister-id 2}
 {:id 2, :brother-id 1}]

Yes. Imagine that we are developing a re-frame app. There is a directed graph of values being computed in the subscriptions. Now, imagine that we want to tell the server about this graph, so that it plays a role to update its values.

> There is a directed graph of value being computed in the subscriptions I do not understand this sentence. Can you provide some [pseudo]code?

I don’t have code for my example.

(let [c (+ a b)
      d (* a b)
      e (- a d c)])

Consider this:

(reg-sub :x
  ...)

(reg-sub :a
  :<- [:x]
  ...)

(reg-sub :b
  :<- [:x]
  ...)

(reg-sub :c
  :<- [:a]
  :<- [:b]
  ...)

Let’s imagine that this is the graph I am talking about.

Yes, that’s what I mean.

OK. So what is the problem?

any graph can be represented by a list of nodes and a list of edges. a list of nodes and edges can be represented by a tree:

{:nodes #{:a :b :c}
 :edges #{ [:a :b] [:b :a] [:a :a] ....} }

The problem is: if I want to describe the usage of the subscriptions by the views, I cannot do it with a tree. I cannot create a EQL structure and tell the server “this is what I am currently using”, because some nodes will be duplicated while in reality I reuse those nodes (that’s the difference between a tree and a directed graph: some leaves can be joined into the same nodes).

So, as @smith.adriane said, I will have to transform the graph and use a custom format. Not EQL or GraphQL.

And when it's a bit unwieldy, you can just encode edges within the nodes themselves:

[{:id :x}
 {:id :a, :dependencies [:x]}
 {:id :b, :dependencies [:x]}
 {:id :c, :dependencies [:a, :b]}]

I am not complaining, I am just sharing the anecdote .. because the name is “GraphQL”

my point is that you could construct graphql query that represents a graph

@vincent.cantin Hold on, again. :) You want to "tell a server" about something. Why do you want to use a query language for that? But even with a query language, it's still possible, although it wouldn't make much sense IMO. Every map in my vector above can be represented with EQL.

FWIW, if I had to deal with the scenario above, I would just create a POST request (or a WebSocket message) and feed it with the relevant EDN data, that's it. Regular HTTP API. Not REST, not GraphQL, not EQL, not whatever else - just plain old HTTP API. If I really had to use e.g. EQL for that, I would use mutations: https://github.com/edn-query-language/eql#28-mutations And it would work in exactly the same way as with the regular HTTP API, only the actual data would be wrapped in some fancy EQL just to make the query parser happy. Same works for GraphQL.

I am brainstorming on the design of a very different web framework, and exploring ideas. Some of what I say may not make sense - that’s normal.

GraphQL represents a graph query that returns a tree

so in that regard, the underlying data source can form a graph, and graphql can easily query that graph. but the return value is a static tree, because that’s what we can serialize and send across a wire

ex.:

query {
  user(id: 1) {
    name
    friends(id: 2) {
      name
      friends(id: 1) {
        name
      }
    }
  }
}

there's a recipe for serializing recursive data structures, this isn't graphql, but the same idea applies: https://netflix.github.io/falcor/documentation/jsongraph.html#reference

> Using References, it is possible to model a graph in JSON.

yep, falcor does have a way of serializing it but then you need an interpreter on the frontend to realize the graph. that’s a tradeoff.

using an actual graph data structure on the frontend also has some tradeoffs, like the inability to console.log it effectively

IMO GraphQL made the right tradeoff: return a plain JSON tree so that any frontend can construct a query & get the response with plain fetch

sounds good. my point wasn't that it's the best solution for every problem, it's that we already have a recipe for representing graphs that some users have had success with

What I don't understand is, why would I use graphql when there is datalog?

or falcor. In 2015 I evaluated both and we decided to use graphql, we have gone trough all what you have discussed above and in the end I still had to do lots of pre and postprocessing on the js side to deal with the results I got from our implementation of GraphQL. Not to mention all the non-standard extensions to it.

Falcor appeared to me back then as something that was started and then abandoned, or at least the public stuff was not very ready for usage at all.

the problem with datalog is that it’s not amenable to front end caching

graphql (and to a certain extant, eql) can be statically analyzed to determine what entities your query returns, and use that to track whether to use the cache & subscribe to changes to those entities

you basically use typename + ID to figure it out, e.g.:

query {
  user(id: 1) {
    ...
  }
}

if you have a front end framework that uses a cache like this, you can skip the request entirely if you already have data

is helix considered a frontend framework? 🙂

I know react is not one.

helix is just a wrapper for React

I’m referencing something like apollo-client or facebook’s relay

using a cache like this also allows you to subscribe to queries. so you can use the query above to subscribe to any changes of User w/ id: 1

Right, so my problem is that all this is conditional that 1. I can implement what is necessary 2. it's easier to implement this than add caching to datalog

regarding 1. I can, but it was hard last time and no one said anything since (in fact, since then I met ten times as many companies who wanted to ditch GraphQL than those who wanted to adopt it) 2. I don't think it's that hard to add caching to datalog, you just need collaborating environments on both end, with deterministic hashing for values. Then you can hash your queries and responses to the queries and only send the diff.

all of this is pretty much impossible to do with general datalog because datalog has certain dynamic properties which prevent you from statically analyzing it. e.g.:

'[:find ?name
  :where
  [?e :config ?name-attr]
  [?e ?name-attr ?name]]

but I do not need to statically analyze a query to cache the results to it

you do if you want to relate the query to entities within the cache, to know whether you should fetch or use the cache

I am not sure what you mean

I don't 'fetch or use the cache' I always fetch. The cache only sends what's necessary.

you can do really simple caching that says “the exact same query will yield the same data”, but you can’t do something like: Query 1: I want the user 1s name, age, and phone number … around the same time, another query … Query 2: I want the user 1s name & phone number w/ GraphQL, you can easily return the cached value for the second. w/ datalog, it can be difficult to know that these two queries are about the same thing

of course you can do that, why wouldn't you be able to do that?

I assume the queries come from components of the application?

because of the reason I said above: datalog is too dynamic to do this for any query

With GraphQL and the scenario that you describe, how do you guard against situations where between Query 1 and Query 2 the value of e.g. the user 1's phone number was changed?

there is probably a timeout or something

@p-himik you can control this. if you want to fetch fresh data you can always request it. but if you’d rather use stale data, you can do that too

I do not do it this way. I do not keep a dedicated cache on the client side separate from client data. The cache is online and sends only updated values. I always send full queries and last timestamps so that the server can know what has changed since.

more importantly is that you can know that new data is being loaded. so you can run query 1 first, and then a subscriber to query 1 can get notified when query 2 is fetching fresh data

or it can ignore that if it chooses, but it will get updated w/ fresh results once query 2 resolves

This is exactly the kind of programming I want to avoid. Having to think about which query is being loaded when.

it’s all about controlling the UX based on what data has already been loaded. e.g. you could show part of the page while some of the other data is loading, if you’ve already populated the cache with some of it

but I do not want to control the ux based on the data, the data already controls the UX, no special wiring, no interfacing, no additional metada over this is required?

anyway, my point is that datalog can run e.g. arbitrary functions in the query. it can dynamically choose keys to use based on other parts of the query. it’s too dynamic

and my point? 🙂

so you can’t use as simple of a solution as reading the query and knowing what entities it relates to

because yours was definitely expressed clearly several times already

you’re basically saying, “the complexity you’re talking about doesn’t matter because it doesn’t match the way that I want to make my apps”

which is fine to say for yourself, but I’m talking about a general application solution a la fulcro, relay, & apollo

I have needs for my applications that using datalog does not meet

I’m trying to explain the constraints which prevent me from using it

and you’re saying they don’t matter. so excuse me for not directly addressing your point

let me give you a tip. every time you consider saying or writing "you are basically saying" to stop and think if the person you are talking to does really say that

because that's absolutely not what I was saying, not even close

What were you saying? Because I've been following the discussion and so far what lilactown has said is completely in line with my own understanding of what's going on.

What he said is not out of line with my own understanding either.

then I don’t understand what you’re saying. > I do not do it this way. > This is exactly the kind of programming I want to avoid. > but I do not want to control the ux based on the data there is either a misunderstanding on my end or yours. I’m trying to explain use cases which I have encountered in my day-to-day, and your rhetorical method has left me with the feeling that you don’t understand or don’t care about them enough to the point where you think that it’s not worth talking about. if datalog matches your use cases well, then you should continue to use it. I will continue to not use it, because it does not match mine.

you are quoting wrongly

the last line requires the part where I say 'it already does that', that's why I don't 'want' it, since wanting something that is given already by default is impossible.

I do not want to cast doubt on your use cases. Nor do I want to change how you work. If anything, my part in this discussion was first to ask why GraphQL fits your needs, to this you said things that I understood implicitly in a way that I questioned them. So it's one thing to state 'my day-to-day' and another to say that 'datalog is too dynamic'. I cannot question your day-to-day, but I can question that it's not possible to write the same thing you do - but differently - with datalog. I understand completely why doing the same thing the same way wouldn't work with datalog, you really overexplained that. Maybe I missed it in the long discussion, but have you touched on why you need to have a separated cache on the client side? Isn't it the case perhaps - and pardon me for being egotistical enough to guess - here: that you have been given the tools first then you had to work out the solution? If you already use all the things required for GraphQL like Relay or apollo (or similar), if you already have a UX toolkit that falsely assumes that everything requires a cache and a fetch and some loader, then all this can just happen without notice.

there’s too much nuance to what we’re talking about to boil it down to what you’re saying. different applications (or parts of an application) may need different behavior. e.g. if you click a back button, I may want to show cached data. if you navigate to a page by clicking a link, I may want to show some of the data that’s cached but also fetch new stuff. OR I may want to show a loading spinner until all data has completed. this requires knowledge at the call site of a query of what specific behavior you want and the action that brought it there. it’s complicated. to support all of these different behaviors, I need a way of knowing what entities a query refers to before I fetch the query from the server.

yes in order to maintain consistency across the UI, I need a front end cache. even just the use case of: Component A renders & fetches some data Component B renders & fetches the same data Component B re-fetches and renders with new data unless I’m okay with Component A and Component B being out of sync, you need a single place to put data that Component A and B can both refer to

they all read the same state, they can't be out of phase unless I am doing some react experiments

so that state then is your cache. it doesn’t have to be outside of the React tree

It's probably worth noting that I completely agree that this is nuanced and complicated.

But my attitude after this realization is to 1. avoid general solutions 2. avoid solutions that require those who work on the visuals and interactions to think about network requests and data fetching

and no, the react state is not the cache, that would be the register

my cache is the same cache that was always the cache, sometime the website can't handle the requests and stores the calculated responses. this is what we called the cache, not the one from the cpu analogy. This cache is definitely nothing like react state.

"Cache" is a concept, not a specific place. Arguing semantics is not very productive.

anyway, thanks for the answers. I now understand that you have decided first that you need a client side cache and then used a solution for this problem. I have no arguments in this area - obviously.

Yes, I do realize that arguing semantics is not productive, however we have already been in an unproductive discussion and realizing that we meant slightly different things by the same word, in this case making what we meant by it clear helped to make me understand the other side much better.

Hey guys! Not sure if this is the right channel for this but wanted to get some thoughts. So I happened to see this Clojure question on Stackexchange via Zulip. And thought hey maybe I could give a shot at answering. Okay, wanted to actually comment at first, given that the qn wasn't too clear ... but couldn't cause I haven't accumulated enough rep >< So granted, I ended up giving a short half answer/half question in my original response which okay fine I got 1 downvote for. But because of the comment not to ask questions, I rephrased my answer and gave quite a bit of due thought to trying to tackle it? But, I ended up finding that 2 people deleted my answer? 😕 Maybe it is because an answer can't be provided before clarification? I don't know? Just me trying to rationalise if I am justified to feel a bit angry or annoyed. Any thoughts? 😮 (Also should I undelete my answer? Also moved this from #clojure ) https://stackoverflow.com/questions/63642728/how-to-convert-from-a-string-into-an-argumentfunction-using-clojure?answertab=active#tab-top