Clojurians Log v2

Um, this idea is fascinating. I read about it last night and I’m still thinking about it. Good work guys.

Quick question: what’s the difference between [?edit <- [?e :todo/edit ?v]] and [(<- ?done-entity (entity ?e))]?

Are they just two ways of writing the same thing?

Also why does define use :- and rule use => as a separator?

The documentation for Precept assumes you already know Clara, and the documentation for Clara assumes you already know rules engines. Is there a good high-level, moderately technical overview of rules engines somewhere?

Another quick question: Clara rules uses defrule for defining a rule, whereas Precept uses rule. What’s the reason for dropping the def, considering that there are other macros like defsub that follow the convention?

@weavejester Sorry. We’re working on documentation. Thanks for bearing with us

[?edit <- [?e :todo/edit ?v]] means bind the whole fact to ?edit that matches the condition ?e :todo/edit ?v

The <- arrow function is semantically the same. In the snippet you provided it means bind the result of (entity ?e) to ?done-entity

The function version allows a context in which to do some macro magic under the covers for you, so it expects a “special form” in the DSL. Right now that is either “entity” or “entities”

entity will grab all the facts associated with a given eid

entities is just the list version of that. So it takes a list of eids and will return you the entity for each one in the same order as the list you provided

Ah, so [?done-entity <- (entity ?e)] wouldn’t work?

Correct

And honestly I think we should be able to support that one because there’s not that much going on underneath the covers

Yeah, I was about to ask if you could; one syntax is easier to remember than two.

That one just expands to [?done-entity <- (acc/all) :from [?e :all]]]

Ahh

Out of interest, what prompted you to use rule instead of defrule?

At the end of the day probably because I thought it was prettier

My justification was after writing your 15th rule you probably know it interns a var and it’s not important to communicate it with the code at that point

Certainly after the 100th it felt that way

Does the idea of an anonymous rule make sense?

Sorry for all the questions

I’m not sure. Clara requires rules to be interned in the ns

No! My pleasure. Thanks for your feedback

This morning on discord we were talking about supporting the ability to write just a LHS and reuse that across multiple rules

Cool. I just don’t want to make it sound like I’m second-guessing your decisions. It’s more that I want to understand them.

Then when I understand them… then I’ll second-guess you 😉

Oh. Well if you are second guessing them I welcome that. But thank you 🙂

Lol

Please do. I like learning 🙂

Does rule support docstrings like Clara?

…don’t know. It should

I think I tentatively prefer defrule, purely from the point of view from someone coming at the code for the first time. There are a few things in Precept that are not entirely obvious whether they intern at first glance. session is another one. I guess that interns a var?

It does

Does session always take a namespace?

Clara has mk-session which does not, defsession which does. Their Clojurescript version has defsession only, but they’re either close or already have a Clojurescript version of mk-session as of about a week ago I’ve just not had time to check it out

Yes

Could you make the def more explicit /and/ less redundant by having a defrules form that takes in a seq of rules?

Or rather, a seq of rule-definitions

Not a seq of anonymous rules

I’m not sure. Clara reads from the namespace and I’m not sure we could supply a list of actual rules in Clojurescript. I think you can in Clojure but not Clojurescript

Thanks for the info. Again, I kinda like defsession over session, as it makes it clear what it’s doing. I don’t think I’d mind the additional three characters… but maybe my opinion will change if I get used to it. These are just initial impressions.

On the subject of first impressions, why does define use :- and all the others use => as a separator?

You’re making good sense to me 🙂

Clara uses =>. rule is a Clara style rule. Prolog uses :- and define is a prolog style rule.

Of course that’s silly. But there you go 🙂

The syntax for the rules is the same, right? It’s just the consequences that are different?

The order is reversed so I think that was taken into consideration as well. Obviously that could mess up some rules

Whether it ultimatley helps people see that quickly or remember /shrug

Ah, so define is “backward?” consequences first then rules?

Yes. Sorry. Should have been more clear

Maybe <= ? 🙂

Lol

I guess that might imply you could use that operator in both senses, though.

It’s a tip of the hat to Prolog though 🙂 https://en.wikipedia.org/wiki/Prolog

Yeah, and that makes sense if you’re familiar with Prolog…

The idea is the same. Rules are “when then”, Prolog is this is true if/when

(rule todo-is-visible
  [[_ :visibility-filter :active]]
  [[?e :todo/done false]]
  =>
  [:db/add [?e :todo/visible true]])

Would something like that make sense? So if the consequences evaluate to a vector, treat it like a Datomic transaction

Might be a dumb idea 🙂

I don’t think so. Fan of Datomic 🙂

We are doing a lot of work inside insert! for schema maintenance. Also, insert! is like Clara, so it’s insert logical. There’s also insert-unconditional!. Both behave the same as Clara with respect to truth maintenance. Precept enforces a schema inside those functions though

Okay 🙂

Anyway, I’ve yet to use this in a real project. I was going to try it on a webchat tutorial I was putting together. Those were just my initial impressions 🙂

At the risk of bikeshedding, I prefer Clara’s defsession and defrule, as that makes it very clear that they’re interning vars. Or if you’re not going for that, defsub kinda feels a little inconsistent?

define I think I could get used to. 🙂

Haha. Yes. That bothers me too

I guess I’m imagining a land where basically there are only rules. I think really we’d like to have everything be (define) or some version of rule that doesn’t even require you to name it. We’ve made some progress on that but it got a little tricky and I kind of just ran out of time

So yes. I think I have a complex about how much Precept is basically Clara. They’ve done all the hard work for us, we’re adding sugar. So I was somewhat reluctant to take the way they named things too 😕

That makes some sense. derive is global but doesn’t def. But I think if you’re interning vars, or adding anything to a registry (like spec’s clojure.spec/def), then def is a convention that has a lot of weight behind it. Those three characters tell those new to the library a lot.

In terms of Simple vs. Easy, it’s “easy”, in that it’s something familiar, but the cost isn’t onerous IMO.

Yep. I knew I was breaking the Clojure convention

That said, maybe I’ll change my mind after the 25th rule I write 😉

Lol

I’m probably pickier than most about syntax, though! I’m currently writing a small websocket library because I couldn’t find one I liked.

Yeah. Not sure if it’s coming across but…We really only want to deal with rules, or things like rules,. Like prolog it’s almost like we don’t even want to write rule…we know we’re writing a rule, that’s all there is 🙂

I hear you on that one heh. And yeah…I’m the exact same way

I forget which websocket library I settled on…

sente

I think if the rules were anonymous it would make more sense, like derive. Or if the rules weren’t assigned to a registry, but were data you could pass around. But there are limits to what can be done with that.

I looked at Sente, but got picky about its syntax 🙂

That all said; there’s a lot that looks very promising about Precept. I’m looking forward to using it!

Thank you. Can’t tell you how much I appreciate your feedback

I’m giving you my initial criticisms, but everything else I like.

That’s good to hear also 🙂

I think you mentioned in your README that you wanted to use it for games as well?

I found when writing Ittyon that an EAVT architecture was good in that case. Knowing when a rule occurred is important in realtime games. But maybe the EAV facts are more general-use.

Yes. Games and game-like UI is one of if not the major motivation

Another random idea: maybe some kind of index that updates according to rules.

But we do a lot of UI work at our company and we’ve always wanted programming to be more like this

In Ittyon, each time a fact is added to the database, indexing functions can be registered that construct data structures for fast referencing.

Hm. That sounds a bit like Rete?

So by default it has eavt, aevt and avet indexes, but it’s often useful to add in more specialised indexing. e.g. proximity lookup based on quadtrees.

Ah! Ok

I think there’s a limit to what one algorithm could do, even one like Rete…

Anyway, that’s just something I found with Ittyon, which is a EAVT-based database for game dev.

It’s rule engine is far simpler though - just reactions to single rules.

Precept/Clara are far more sophisticated in that respect.

There is. Hence quadrees on the roadmap, other general purpose algorithms. Our thinking was to try to use implementations of those algorithms out of the box

It would be nice if you had a way of hooking in custom indexing.

We do have eav and aev indexes in the schema layer for enforcing cardinality and uniqueness

IMO indexing is super important for games based around fact databases.

Ok. And you’re thinking for performance or convenience? Both?

Performance.

I found most performance problems were related to indexing.

But I’m just one datapoint 🙂

Or could be solved by better indexing.

There’s certain situations where Clara/Rete don’t give us that, but in most cases it does in a pretty optimal way. Especially for games where you’re dealing with a lot of incremental changes

Incidentally, Precept might hook into Impi well: https://github.com/weavejester/impi

Oh nice!

Anyway, thanks for answering my questions and suffering through my criticisms!

I need to go eat, now!

My pleasure. Thank you 🙂

> Precept models state as a graph. We can add new facts about the world without concerning ourselves about its location in an object. We can query data and perform derived computations on it just as easily. Is there any thought to adding the option to expressing the apps state transtions as state charts. possible using statly? https://github.com/nodename/stately

Does that idea even make sense? 🙂

Hi. Not sure. You’re thinking as an implementation or a tool to visualize state changes?

@alex-dixon The way i understand precept is Precept is a rule engine where: if condition then apply rule. But what if the condition is dependent on other conditions being true? So it depends not only on the current input, put past inputs and what state were currently in. I believe its possible to model this as a FSM. But apparently its very cumbersome. StateCharts are an abstraction overtop of FSM to allow for some re-use. So it occurred to me that you could build a StateChart model for Precept

@drewverlee So as something that would help a developer understand their app as they were making it?

@captainlexington At this point i’m just wrapping my head around What Precpt is, i’m vaguely familiar with the idea of a rules engine and i’m trying to work though how you would structure an application using it. The first hurdle i had was trying to see how you would achieve different results depending on pre-existing inputs to the system.

@drewverlee oh ok. I think you'll find Precept handles that case rather well. 😊

You should be able to insert facts that are preexisting, no?

Rules have no ordering. Does this answer your other question?

Also a rule may have multiple conditions

We're working on docs ATM. In the mean time you might check out Clara's examples to see a given rule can respond to facts that may have been inserted by a chain of rules

https://en.m.wikipedia.org/wiki/Forward_chaining

I also agree with @weavejester on the def convention. It wasn't immediately clear to me that rule was interning a var until I looked at the impl. But again, I am just now taking a look at this project so my opinion may change as well 🙂

Also I think the example project would be much easier to read if vars weren't :refer'ed and an alias was used instead.

@kenny Valuable feedback. Thank you 🙂

Curious about the relationship with statecharts too. I’m not familiar with the clara rules engine, but it seems that rules would be a great way to express transitions between states. What seems to be missing afaict is the ability to dynamically load rules as the state changes?

@jfntn Correct. If statecharts could solve that we’d definitely be interested in taking a look. It’s very likely we will want different rules to be loaded in different contexts for performance reasons

Left hand side calculations may run in some contexts for which they are not relevent as an artifact of Clara’s implementation that allows parallelization

Right, the current state should determine what rules are running

I think you mentioned earlier that this isn’t yet possible for clara in cljs, is that correct?

@jfntn Yes, Clara is fairly eager in its processing (from what we can tell) so if you have rules in your session they will do some work even if you gate them with a controlling fact (e.g. make the first pattern something like [_ :current-state :firing]. Clara has a concept of activation groups but currently those control salience and do not gate processing. Other rete-ish algorithms (like Drool's phreak algorithm) are much more lazy and can be gated performance-wise so you can keep all the rules in one session and still have strongly bounded contexts.

(I'm at CoNarrative with Alex btw)

And yes rules are definitely a perfect way to transition between states.

So one way we're using Precept today is keep the "current state" of a finite state machine (more or less) in one global app session. We use file structure to bundle edge transition decision rules how we want. But all the rules for the various states are already compiled in the one session; just not activatable untl you're in that state.

So the "current state" does effectively determine what rules are running -- there just may be some background overhead.

So far that hasn't been a problem for us. But we're keeping an eye on it! If it does, we have a couple of future strategies in mind. The easiest is to maintain multiple sessions each with their own set of rules. Say at the "top route" level. On transition, we'll just "pour" base/ground/underived facts from the current session state into the new session. Within that context now only the relevant rules will be fired, but it'll have "memory" of its common state.

A more elegant way will be to work on it at the Clara engine level. Kudos to them by the way -- none of this would be possible without their work!

@weavejester : We’re actually using Precept as the basis for a web app that has very dense, visual, game-like UI with lots of real-time decision logic (along with lots of more conventional web UI). Think live UI decisions during a drag operation with potentially many, many elements on canvas. (btw @jfntn we model the various states of the various mouse operations as state machines and use rules to transition between them.)

To do some of our collision/intersect detection efficiently, we are looking at using quad-trees (and r-trees because much of our data may be irregularly-sized & nested) to inform the rule engine about proximity. In that use case, the relationship between “quad-tree” and Precept is somewhat like between a game engine and a Lua-based scripting language; the scripting engine isn’t necessarily doing all the heavy number-crunching but receiving synthesized inputs over which you can express complicated logic.

Have you guys given any thought to best practices when interacting with external resources (e.g. a server). The obvious way would be to have a rule execute when a certain interaction occurs (e.g. user clicks a button) with the RHS sending a HTTP request to a server and insert'ing or retracting based on the response.

@kenny Yes, we're working on some examples along those lines right now!

In short, the response is always going to insert-unconditional or retract

how to trigger the HTTP request itself (the side-effect) we are of two minds. The simplest way (and what we're doing today) is what you just said: fire from the RHS/consequence.

A more purely functional way would be for the RHS to just insert a fact representing a side-effect intent (like a HTTP request). That way fire-rules wouldn't actually trigger the side-effects, but calling infrastructure could process the "side-effect queue" instead. We're not currently experiencing any pain points around the simple "do it in the RHS" way though.

That may be because mostly we build apps that are more 'fire-and-forget' message-driven. Send off some message to the server; one or more future messages get pushed back. The best practice there is simple -- just put them in the action queue like something coming from a UI interaction.

"them" being messages from the server

Right, ok that makes sense. Let me walk through a simple example and see if it makes sense to you. Let's say we are building a login form with an email and password. You will need some subscriptions for the email and password:

(rules/defsub :email
  [[_ :email ?email]]
  =>
  {:email ?email})

(rules/defsub :password
  [[_ :password ?email]]
  =>
  {:password ?email})

(defn Login
  []
  (let [{:keys [email]} @(precept/subscribe [:email])
        {:keys [password]} @(precept/subscribe [:password])]
    [:div
     [:input {:type      "text"
              :value     email
              :on-change #(precept/then [:global :email (.. % -target -value)])}]
     [:input {:type      "text"
              :value     password
              :on-change #(precept/then [:global :password (.. % -target -value)])}]
     [:button {:on-click #(precept/then [:global :logging-in? true])} "Login"]]))

(rules/rule
  logging-in
  [[_ :logging-in? true]]
  [[_ :email ?email]]
  [[_ :password ?password]]
  =>
  (do-login ?email ?password
            (fn [response]
              (if (= (:status response) 200)
                (precept/then [[:global :logged-in? true]
                               [:global :logging-in? false]])
                ;; handle errors...
                ))))

@weavejester and anyone else interested in how Precept might interact with external indexes and calculations (physics, quad-trees, etc.): https://github.com/CoNarrative/precept/issues/67

@kenny Yes, that's exactly it 🙂

Technically you currently don't need to use precept/then in the consequence and can just use insert-unconditional -- but using precept/then is better future-proofing as it queues it up for next tick and we might enforce that flow at some point.

Also, an alternate is to insert the status & data directly into the rules, and then have other rules do the handling of each possible success/error case.

Basically, any time you see an "if" in the RHS/consequence it might be a signal to insert for downstream rule processing (you'll have lots more logical flexibility as it's what rules do 🙂 )

Right, I was thinking that but this is all new to me so I was unsure.

BTW do :global and :transient have a special meaning for the e? I see it used in the example but it's not clear what they mean.

:global is not treated specially and just a convention to say "hey, I'm a singleton!"

You can make up whatever entity ids you want

:transient does (currently) have special treatment. Any eav tuple with an entity id of transient gets cleaned up at the end of a fire-rules

Gotcha. Technically you could use any keyword but :global is just the convention.

Cleaned up means retracted?

yes, sorry 🙂

:transient itself is a very lightweight concept and consists of a single rule:

(clara.rules/defrule clean-transients___impl
  {:group :cleanup}
  [?fact <- :all (= :transient (:e this))]
  =>
  (clara.rules/retract! ?fact))

Nice and simple.

Someone could call it :action or :command (that might better communicate what it's for) and add the same rule for that entity id, or if you wanted to queue commands in the session you could generate separate entity-ids for each command (and mark them in some way for cleanup, or keep them around for a command history, etc.)

This leads to my next question of the meaning of {:group :action}, {:group :cleanup}, etc. At first I thought they were for documentation purposes but now it seems like it has some effect on rule ordering?

yes, that's from Clara and they're called "activation groups"

In precept.core:

(def groups [:action :calc :report :cleanup])

default rule flow goes in that order -- handle mutations from incoming actions, derive calculations, hydrate maps for the view, then cleanup transients

(the action parameters that just came through the pipeline)

You can roll your own flows too -- that's just the default we provide. In Clara, activation groups don't prevent rules from firing (like "agenda groups" in other rule engines which act as states in a fsm). Clara activation groups just control salience, that is, which rules are going to be fired first.

Gotcha. Counterintuitive name.. 🙂

If I insert a fact that look like this [:transient :my-map {:foo 1 :name "Bob"}], is it possible to write a rule that uses the map? Something like this:

(rules/rule my-map-one
  [[_ :my-map ?m]]
  [[?m :foo 1]]
  =>
  (println "One!"))

@kenny rule engines (Clara certainly upon which we're based) typically model everything in a flat/relational graph manner instead of using nesting. If there really are child/related entities, insert them separately and join. So one way to express the above would be as a "join" between the :transient entity and the :map entity:

[:transient :my-map 43]
[43 :foo 1]
[43 :name "Bob"]
[43 :transient true]

or, because I think I want :transient renamed (or just a convention)...

[:this-command :my-map 43]
[43 :foo 1]
[43 :name "Bob"]
[43 :transient true]

Totally makes sense. Are there any utility functions to make this easier? Or is this not a common enough pattern?

if that latter makes it clearer. The :transient attribute is just there to mark the related entity for cleanup (you'd have to add a rule for that, but simple enough)

No, it's common and we have some helper utilities. I'll defer to @alex-dixon on those -- @alex-dixon ?

precept.util/tuplize-into-vec looks like it

(precept.util/tuplize-into-vec {:db/id 1
                                :a "a"})
=> [[1 :a "a"]]

I'm wanting as much inter-op between maps and eav tuples as possible - just two ways of looking at the same thing and sometimes one is better than the other for a particular case. The core representation will remain eav tuples under the cover of course.

And that makes sense. I suppose I was trying to mimic Datomic's tx-data. i.e. You can pass datoms or a map. If passed a map, then Datomic will turn it into tuples.

https://github.com/CoNarrative/precept/issues/28

😊