Clojurians Log v2

Hello all, I've been trying a bit of clara-rules to assess it for a project. I'm having some trouble with with-tracing though.

@dave.tenny uploaded a file: https://clojurians.slack.com/files/U7SGKB4LF/FAVQH9N22/-.clj

If I omit the with-tracing call fire-rules does its work. If I add with-tracing before it, fire-rules no longer works (none of my rules are fired).

If I move with-tracing before (insert ...) I get lots of traces.

What am I doing wrong?

I'm using [com.cerner/clara-rules "0.18.0"]

and clojure 1.9

I don't want to trace the rete activity on the initial insert, just the activities of fire-rules and the working memory alterations there.

@dave.tenny it’s an interesting point you bring up, but I don’t think it really makes sense to only “partially” have with-tracing enabled

All session interactions prior to fire-rules are susceptible to lazy evaluation and batching.

In general, I’d stress to not rely on any stateful information of a session that has had inserts/retracts performed on it, but no fire-rules

Heh, not sure I was trying to make a point. I didn't understand why fire-rules didn't work based on the placement of with-tracing. Since my rules weren't firing.

I’d actually rather it be the case that the API threw an exception if you tried to query or do something like with-tracing when it was in an inconsistent/lazy/batched state prior to fire-rules

Naively (since I am naive here), I just figured my session would be upgraded to do tracing.

It’s an interesting point regarding the limitations of with-tracing and lazy/batched insert/retract behavior I meant

I certainly haven’t dug into details to diagnose your specific issue, but I’d imaging the with-tracing added late, after the insert missed some listener information it needed

So how should I be using with-tracing here?

Only add it when you have a session that doesn’t have insert, insert-all, or retract calls possibly still pending on it

so either (a) right after your mk-session, or (b) right after a fire-rules

I'm doing insert! and retract! in my rule RHS's, but but the only insert was as shown, before the with-tracing call.

but you won’t get tracing “only for the fire-rules actions that took place, you’ll get a complete picture, you can looking into tracing details to find out specific things, like facts inserted due to rules firing etc (at least to some degree)

@dave.tenny you did this?

(-> (mk-session)
	(with-tracing)
	(insert ...)
	(fire-rules))

Yes, that way I get tracing, and fire-rules works as expected.

that’s the way to use it

ok, thanks

So I don’t know what your question is beyond that and beyond what I said about it being misleading in the API to let you add it at arbitrary points

I'm trying to get the network activity after the initial bulk load of facts.

I think it’d be better to reject an attempt to add it at the “wrong times”, enhancement

And when I do that, by moving the trace after the facts are loaded, my fire-rules don't work.

yeah, it doesn’t work on that principle

ok, good to know, still learning

That begs the question of when it is safe to use it. Say I make a session, load data, run rules, query results. Then I want to change the data in the session and run it again with tracing. Will it work then? Or is it always safe to use with-tracing after fire-rules calls?

I’m not actually sure there is a way to look at the listener information and know the different between a fact inserted externally, via insert and a fact inserted from a RHS, via insert!. I don’t know of one off hand.

> Or is it always safe to use with-tracing after fire-rules calls? I think so and expect so. If it has issues, it’d be interesting to discuss them. Also, I don’t have any reason to think that it should ever change the semantics of the rules, you may just get bad/invalid tracing results

Earlier you said it caused your rules to not fire correctly, I’m skeptical on that

the tracing may not have accurately reported the rule firings, but they probably still fired

you can test that assumption, by adding a defquery to your session

re: firing, my rules had printlnin them, which didn't print when the with-tracing was done after the insert.

and then using a query on the RHS insert! facts that you think didn’t get inserted

Yeah, if you can reproduce that case at all, that’d be valuable

Okay, well it's pretty simple, I'll see what I can do.

I don’t expect with-tracing to be able to interfere with evaluation in that way

@dave.tenny found it

yeah, so with-tracing does appear to wipe out pending work

so you’re right, it’d cause trouble like you saw

after fire-rules should be safe - there is no pending work

it may be a “good thing” that with-tracing wipes out pending work since it makes a session that it can actually trace

well, idk about that, it’s just a weird situation hah - either needs docs or to throw an exception or something I think 🙂

Sounds like the documentation should be a bit clearer about restrictions though to avoid noob run-ins like mine.

Yes, it is too mysterious

I'm just trying to wrap my head around other things, like the Truth Maintenance capabilities that may or may not have been present in the OPS5 systems I was using 30 years ago. I'm pretty rusty at this. Retracting the preconditions that led to the state of the RHS doing the retraction seems to mean I need to fire rules, query my reults on the session afterward, make changes, then rinse-lather-repeat, where it used to be I'd do more stuff in the RHS clauses. Thus my attempts at ... modular ... tracing.

So a bit of catching up for me. I'm investigating clara for the job dispatch logic of a new scheduler I'm writing.

Meanwhile thanks for the help.

sure

@alex-dixon has done a lot with Clara’s tracing by extending it for his own purposes in the #precept library. Perhaps you can find some inspiration there.

https://github.com/CoNarrative/precept

I think I ended up using the listeners namespace only. I forget at the moment how it relates to clara’s tracing

@alex-dixon oh, tracing just organizes the data coming from listeners I believe

wait no, tracing is a default impl over listeners

Ah ok. That what I seem to remember but thought I may have been confusing it with explanations

(sort of the same as I said)

yeah, explanations/inspect are above all that

Yet listeners is very user friendly imho. Though I may have ran into some private things I wanted public

It is nice to have it boxed up and ready to use just at the same time the listeners ns is highly usable in and of itself I mean

@dave.tenny If you end up poking around Precept at all I’d recommend the issue 19 branch

Will keep that in mind, so far have only looked at Clara-rules.

(defquery my-query ...) followed by a what is typically another function wrapping the query with a call to (query session my-query) gets old. Is there w away to do a lexical defquery equivalent at the point I'd call query?

defquery more or less sets up an endpoint in your rules network, it needs to be known before hand

query goes and grabs the list it generates at when things are fired, it doesnt go calculate it then (to my knowlege)

you can, and i would not reccomend this outside of development, create a defquery for Object that just grabs everything in the session and then filter that as you see fit.

@dave.tenny I’ve only partially read your example so far, but one immediate piece of feedback, don’t query until after fire-rules

think of query the same as with-tracing earlier, you have to only do it on “consistent” states of a session

Clara does some delayed evaluation for performance reasons. Any inserts/retracts that occur prior to a fire-rule cannot be relied upon and I’d consider the session to be in an “undefined” state

I’d actually prefer Clara to change for query to either (a) throw exception if called prior to a fire-rules or return something that indicates it is not ready yet.

Clara takes advantaged of batched insertion and also potentially a few other things, so it does not eagerly perform things like insert or retract. The evaluation is typically pushed to batch up the facts propagating through the network. This can play a major role in performance.

You mentioned OPS5, which you likely wouldn’t have seen behavior like this long ago. It is more common now. Drools is a popular JVM-based rules engine and it similarly has delayed evaluation of things prior to rule firing due to similar concerns.

This issue seems to somewhat often trip people up though. So I don’t think it is clear enough.

For your second question > ;why are my counters still there, I retracted them? I think it is because (query % counters) doesn’t return what you are thinking it does

It doesn’t return the bare counter fact, instead it returns a bindings map from the query, so it returns something like:

{:?counter <the counter fact>}

so you’d probably have to do something like

(#(apply retract % (map :?counter (query % counters))))

Oh sorry, forgot to include the output

It works, just not as expected

Here comes the output

that output looks consistent with the explanation I just gave

so try to apply the things I mentioned

Ah, that fixes it

(map :?counter ...)

🎉