Clojurians Log v2

I'm diving into the illustrious and awesome Biff framework. I'm not sure how I should be connecting to the terminal based repl that runs, in that framework, via ./task dev. I tried cider-connect-clj&cljs, chose shadow, then :app. At that point: • I can evaluate a form in either repl buffer • I can evaluate a form in any clj buffer (via cider-eval-defun-at-point) • I can evaluate a form in any cljs buffer (via cider-eval-defun-at-point) if I switch to clojure-mode • However, if I evaluate a form in a cljs buffer using cider-eval-defun-at-point nothing occurs, either in the buffer or in either repl buffer I recall having this issue before several times in non-Biff projects, so it's likely how I'm going about it all. There are a myriad of permutations of how I could be connecting from Emacs... maybe I should be trying a different approach. Any thoughts or things to try?

If I recall, using cider-jack-in-clj&cljs often would solve this sort of problem in the past, but here I think I need to actually connect because Biff's ./task dev is doing other things. At any rate, doing so here produces

error in process sentinel: Could not start nREPL server: shadow-cljs - config: <path-to-project>/shadow-cljs.edn
shadow-cljs - connected to server
server already running

I just noticed that, whilst in a cljs file, when in clojurescript-mode I see

cider[not connected]

But I do have a working cljs (as well as clj) repl buffer in all cases.

this might be a longstanding issue. when in the cljs buffer, focus the cljs repl and then focus the cljs buffer again. do not go into the clj repl. it only "sees" the last repl you were in

Ah, yes. That has worked in the past. Here, unfortunately, it doesn't. 😞

actually, hang on while I monkey around with variations on that.

Well, that fix not working this time.

But it's good to remember that one; it's helped in numerous occasions.

The project team primarily use piggieback and Vim, if that's any help.

the cljs repl says 'REPL[pending-cljs]' in the status bar, but does eval forms.

Can you eval (js/alert “hi”)?

I’m guessing it will say unknown namespace js

If so you need to connect to the js runtime as it’s still a clj repl. Load the webpage or run the node script

yep...

Caused by: java.lang.RuntimeException: No such namespace: js

Yeah. Is this a web app? If so load it up in the browser

You need a js engine

It is a web app... it is loaded in the browser... I just reloaded it without success.

Is figwheel on your class path ? As a dependency somewhere

let me grep for that.

So to step back, you need an nrepl server that has cider and piggieback involved so that it can handle the js stuff for you

okay...

searching for piggieback... the project creator is a Vim user, so I don't know if that completely crosses over... https://github.com/jacobobryant/biff

I should just use Vim, but I'm so... addicted... to Emacs.

He does evaluate from within a cljs file in Vim in one of his demonstrations, for what it's worth.

you can definitely use piggieback + emacs

whew.

If it works for him in Vimland... does that imply it has what I need as an Emacsoid?

It seems like he'd also need piggieback... but that's way over my head at this point.

I use piggieback+emacs+cider and it works wonderfully for me

but, you mention that

you have some clj buffers and some cljs buffers

I do.

and, while I understand that this does work for some people, I haven't used it that way

I'm generally writing one or the other, in a particular instance of emacs

things eval in the clj and cljs repl buffers, and in clj files, and clj will eval in cljs buffers when in clojure-mode...

I think it should work fine with 2 nrepl connections

yes but when you are setting clojure-mode in your cljs buffer, are you sure it is evaluating clojurescript and not clojure?

Caused by: java.lang.RuntimeException: No such namespace: js

looks suspiciously like you are sending clojurescript code to your JVM clojure process

I am guessing it is eval'ing clojure.

yeah so

what you want to do is use piggieback

to sort of switch modes...

you can also have 2 active CIDER connections

I wonder where I'd add that in to this framework (https://github.com/jacobobryant/biff) then...

2 repl buffers

I haven't seen this biff framework, I will take a look

because the last I understood anything was in the glorious simple days of project.clj and so on.

Then the clone wars happened, things got complicated, and all that.

Biff is actually phenomenal. But it's what one would call a 'learning opportunity' for a simpleton like me.

Here is his overview page: https://findka.com/biff/#introduction

you are using figwheel; figwheel's cljs-repl supports (expects, I think) piggieback to be on your classpath

are you using leiningen?

is your name a play on Slartibartfast?

I normally do, but here I run the framework's

./task dev

It is composed of simple Clojure idioms, however.

https://github.com/jacobobryant/biff/blob/master/task

it's an extremely short bash script

haha... and yes, my name is indeed a play on Slartibartfast. 😀

I'd rather be happy than right, any day

"Perhaps I'm old and tired, but I think that the chances of finding out what's actually going on are so absurdly remote that the only thing to do is to say, "Hang the sense of it," and keep yourself busy. I'd much rather be happy than right any day."

haha... our messages crossed, lol.

cider/piggieback {:mvn/version "0.4.1"}

^ add this to your ~/.clojure/deps.edn (or to your project's deps.edn)

ok, on it.

sorry that version is not up to date

there are some other dependencies too which you may or may not already have

:aliases {:nrepl
          {:extra-deps
            {nrepl/nrepl {:mvn/version "0.8.2"}
             cider/piggieback {:mvn/version "0.5.2"}}}}

it goes under :aliases?

I believe you need to configure figwheel to use cider/piggieback too

see here https://github.com/bhauman/lein-figwheel#figwheel-server-side-configuration

yes :aliases is one way to do it

Bruce is straight out of Hitchhikers.

it depends if you think that figwheel is a personal development tool

or part of your projects dev dependencies

you can put it in the project's deps.edn

if you want other users of your app to have it for sure

your figwheel config probably wants to configure the nREPL middleware

I'm putting it in globally.

see where it says

Load CIDER, refactor-nrepl and piggieback middleware

piggieback, that is.

without that it will sort of work in a crippled way (lots of CIDER features missing)

ok... trying to get that all in.

I am trying to catch up on what biff is doing when you run ./task dev

but you do have figwheel running, right?

I have a function in my clojure dev namespace that first starts figwheel and then calls figwheel-sidecar.repl-api/cljs-repl

which converts the clojure nREPL connection to a clojurescript one, using piggieback

I don't know if I do...

that is, if I have figwheel running.

ok

I once understood things... barely... then boot came along, then the successors...

is the code something you can share?

or, if I generate a new empty biff project and get emacs+cider+piggieback working would that help?

omg, tremendously.

I would also then be the rather dubious Biff Emacs integration expert, as well.

Presumably I could write it up for the Biff docs.

And eventually add it to the project, for the Emacsians. But I'd probably have folks here audit it first, lol.

I am fiddling about

OK, I'll shut up.

haha, not what I meant

I know.

did you say that you were using figwheel? because it seems like my newly generated biff project uses shadow-cljs

I see nothing indicating that it uses figwheel

aha.

I don't even know what I am using... I connect via cider-connect-clj&cljs from within Emacs.

So I don't know if that implies figwheel or not.

how are you starting the repl?

with this arcane bash stuff in biff ... I am looking into what it does

Well, in this case, the repl is started by the framework...

dev () {
  if ! [ -e node_modules ]; then
    init
  fi
  npx onchange -i tailwind.css -- ./task build-css-dev &
  trap 'kill $(jobs -p) 2> /dev/null' EXIT
  mopt="-A"
  if clj -Sdescribe | grep -q repl-aliases; then
    mopt="-M"
  fi
  BIFF_ENV=dev clj $mopt:cljs "$@" -m $MAIN_NS
}

that's what it does ^

when you run ./task dev

yeah. its crazy. yes. so are you calling ./task dev?

thank you, was just about to paste it.

I am.

ok. i'm not sure how you're even connecting to that. i don't see any nrepl getting started or involved

and, when I did this, I got some complaints about it not finding cider/nrepl and refactor-nrepl/middleware

hang on, let me look.

i'm confused why it even has an nrepl server. that's just calling clj -M:cljs -m example.core or whatever

right

perhaps they bake it in

I'm digging around, but pretty slow on the uptake on this.

when you run ./task dev, this is in the output in the terminal:

Downloading: cider/piggieback/0.5.2/piggieback-0.5.2.pom from clojars

is that because you added piggieback as a dependency in your ~/.clojure/deps.edn perhaps?

ok. so shadow is in charge of the nrepl stuff. but there's no middleware so CIDER can't work with it. wont need piggieback in a shadow-cljs project

so that's not an issue. its just getting shadow setup with the correct middleware

I don't think so... you do finally get

shadow-cljs - nREPL server started on port 7888

and the main ns calls biff/start-system with its own map

cider docs have instructions on cljs jack-in with shadow-cljs here https://docs.cider.mx/cider/cljs/shadow-cljs.html#using-cider-jack-in-cljs

@macrobartfast I thought I understood you to say that you were using figwheel, but I think that is not the case... maybe I misunderstood

I tried cider-jack-in-cljs but had issues...

this stuff is absolutely crazy and i can't see you getting it running with CIDER unless you're an expert

(nrepl/start-server :port 7888) is burried in biff.components and is called by biff/start-system

well, that's helpful to know...

haha this framework looks ... deep ... I am trying to get it to work with jack-in-cljs

it looks cool but yeah, adding cider+cljs is not going to be super obvious

I had thought it was just a matter of choosing the right cider command in Emacs or whatnot, lol...

some things you could do is pass :biff.init/start-nrepl false and be in charge of your own nrepl server. but the challenge is that all of this stuff is kinda twice removed from your own project

so you need to tell it to not do anything and then do everything correctly yourself. but the css is watched in some random shell scripts, the nrepl and shadow started elsewhere, etc

i don't care for this kind of thing. i much prefer you being in control of starting nrepl, and shadow.

(defn init [sys]
  (let [env (keyword (or (System/getenv "BIFF_ENV") :prod))
        unmerged-config (bu/catchall (edn/read-string (slurp "config/main.edn")))
        config (some-> unmerged-config (merge-config env))
        {:biff/keys [first-start dev]
         :biff.init/keys [start-nrepl start-shadow]
         :as sys} (merge sys config {:biff/unmerged-config unmerged-config})]
    (let [start-nrepl (if (some? start-nrepl) start-nrepl (not dev))
          start-shadow (if (some? start-shadow) start-shadow dev)]
      (when (and first-start start-nrepl)
        (nrepl/start-server :port 7888))
      (when (and first-start start-shadow)
        ; Without this, I sometimes get the following when loading localhost:9630:
        ; java.lang.Exception: Unable to resolve spec: :edn-query-language.core/property
        (require 'edn-query-language.core)
        ((requiring-resolve 'shadow.cljs.devtools.server/start!)))
      sys)))

yeah... that's what made me think I wouldn't be able to just jack in, as it were...

here's all the code in biff that does this stuff. so you need to smuggle all your middleware down through to here, etc

gotcha, ok...

I have discovered something... I know why it complained about middleware not found when I started ./task dev

#:biff{:first-start first-start
       :routes routes
       :static-pages pages
       :event-handler #(api % (:?data %))
       :rules #'rules
       :triggers #'triggers
       :jobs jobs
       :send-email #'send-email
       :after-refresh `after-refresh}

it's because, shadow-cljs.edn has :deps {:aliases [:cljs]} in it and that is an alias I have defined globally

nah its because of this line from above: (nrepl/start-server :port 7888) no way to get middleware into there

isn't there ... an edn file

yeah there is

cat ~/.nrepl/nrepl.edn 
;; can be overridden locally with .nrepl.edn ... which replaces (not merges) those here
{:bind "localhost"
 :middleware [cider.nrepl/cider-middleware
              #_cider.piggieback/wrap-cljs-repl
              refactor-nrepl.middleware/wrap-refactor]}

oh yeah. i hate the idea of globally defined files for this stuff. that's system wide and expects random projects to have those deps

I mean, I agree with @dpsutton in spirit... I don't like too much magic around starting the system

so many issues from old profiles.clj people have laying around

but I still think I can make this biff + cider thing work with what's there

I understand where you're coming from, @dpsutton.

but anyways, this is all designed if just connect to an nrepl port with no other requirements. cider will be very neutered in this thing.

This project is an effort to get a more turn-key thing going for people, but it fairly bound to a specific editor, at the moment, it seems, at least for inline eval'ing.

This has exposed my lack of comprehension of cider and figwheel in general... which is a good thing.

It's probably time for me to finally put more effort into understanding the repl magic I like so much.

I kind of understood things when figwheel first came out (yay, flappy bird)... but things got increasingly complex for me to grok.

So... while I am not sure exactly which features of cider+figwheel you loved, I want to point out that shadow-cljs is a very different way of working with clojurescript; I have not yet tried it, but I suspect that dpsutton is correct that a ton of useful CIDER stuff will be missing

it sounds like you can get some of the same features (live cljs recompilation, cljs repl)

gotcha... mainly, I like being able to hit a db from within the code...

as well as atoms...

hit a db ... from jvm clojure right?

so I can refine my functions.

Well, db from Clojure, atoms and remote calls from ClojureScript.

That's my general use, anyway.

ok, good

then you wont miss the rest of the CIDER features

I am still fiddling about

I'd be cool probably without them.

thus far I have not gotten biff to start and have cider middleware

ok... massive gratitude, btw...

happy to try

biff does look interesting... it is far from the way I am used to working with clojurescript though

Biff is a cool project in general, and so if I can get it to work I learn all about crux and so on, the things baked into the framework.

It also allows you to deploy a site to the very affordable Digital Ocean very quickly with a lot of features I want in a basic project... and I like to experiment so it's appealing.

ooh it uses crux? very cool

well I got a cider+piggieback cljs repl working in emacs with my biff project ... but I'm not there yet

I'm as excited as a pomeranian on the back dash of a car who's just spotted its owner come out of a donut shop.

unfortunately I think what I've done is not compatible with the way biff starts the repl though

oddly it sounds like piggieback + shadow-cljs is not really a thing, even though the vim-fireplace code supposedly supports converting to a cljs repl with shadow-cljs, and calls this feature :Piggieback ... I think the relationship is in name only

@macrobartfast do you know if anybody is using biff without shadow-cljs?

I don't think anyone is.

I don't know, really...

but it's a fairly new project.

it is possible that the Right Way to accomplish this (cider+biff) is to provide an alternative that starts nREPL with cider middleware and then uses piggieback to turn it into a cljs repl

the framework creator would probably be super down for that...

that is going to have the advantage of supporting all the amazing CIDER features;

they indicated curiosity about Emacs in general.

the thing is, the selling points of shadow-cljs don't apply to everyone... this is the kind of thing that makes designing opinionated frameworks extremely challenging

IIUC

I mean I have always loved using figwheel and piggieback, and didn't mind that it was running JVM clojure for the cljs compilation. I am not an expert on shadow-cljs at all, but IIUC it is using the self-hosted CLJS compiler

well, the framework touches on one of the Clojure conundrums in general... how to get a greenfield project up and deployed with usual features in a short amount of time...

and I guess I would go to another language for that sort of situation... but, of course, I'm not going to for all the reasons none of us would.

nah I must be wrong about that... clearly shadow-cljs is starting a clojure process here

I don't see anything fundamentally wrong with the idea of having another way to start biff that uses piggieback. I was able to make it work (but not loading the actual biff app's cljs sources)

I mean, I created a biff project, added the necessary middleware, and then I was able to get piggieback to work compiling something in the context of that project

that is really far from the way biff has it working though, with shadow-cljs

pomeranian actually shatters rear windshield

https://github.com/tpope/vim-fireplace/issues/322 refers to some CIDER doc that says that shadow-cljs provides its own middleware so maybe I am going about this totally the wrong way trying to use piggieback

omg this is all deep... reading that now.

I do actually see some evidence that biff without shadow-cljs might be a thing already

I just asked the Biff main author about this.

it seems like maybe the biff SPA only knows how to compile the cljs using shadow-cljs though

biff.core/default-mpa-components…

:biff/start-shadow...

yeah I found that

but without shadow, there's no clojurescript compiler included, unless I am missing something

however I did just get a cljs repl buffer in emacs that was connected to the browser which had my biff app loaded

https://github.com/jacobobryant/biff/blob/58c43851393daa0020f27c407f9b2712944823a1/src/biff/core.clj#L37

but wasn't able to get my emacs clojurescript-mode buffers to eval over this connection

ah ok

I had a cljs repl... but I don't know if it was browser based... js/alert never worked IIRC

I have used cider-connect-cljs and connected successfully to the nREPL server started by ./task dev which is launching shadow-cljs

so I have a cider-repl buffer in emacs, in which (js/alert 1) works

with the biff app in the browser

this is pretty close to what you want

but not quite

I don't understand yet why the clojurescript-mode buffer is not sesman-linked with the cljs repl connection

googling sesman

an emacs dependency of cider

which allows it to have multiple nrepl connections simultaneously, and each buffer associated with one of them

gotcha

this is where the forms go when you cider-eval-last-defun for example (for each buffer, sesman associates it with an nrepl connection)

it seems like it should've been associated after I got cider-connect-cljs to work

with the shadow-cljs option (`:app`)

but it aint yet 😢

right?

all I wanted was a Pepsi.

aha, the slack archives has a clue! https://clojurians-log.clojureverse.org/cider/2020-05-18

perusing that.

but also the basics https://shadow-cljs.github.io/docs/UsersGuide.html#missing-js-runtime

are you trying those newer versions?

bummer ... the symptom I am seeing now is identical to that person in the slack archive

I wanted to update the versions but ran into some troubles; there is presumably a transitive dependency on clojure/clojurescript from either shadow-cljs or from biff itself

I am cleaning up my shadow-cljs situation... turns out I didn't have a top level shadow-cljs.edn, for starters...

anyway that version mentioned in the slack archive as "fixing" (somewhat mysteriously) the issue is old-ish today

gotcha

I don't have a top-level shadow-cljs.edn either

I had one in my Documents folder, lol, from another project.

It found it.

https://shadow-cljs.github.io/docs/UsersGuide.html#user-config

I am learning a ton right now.

ok, have a js/alert working from the cljs repl now myself, which is progress.

it seems like it is very very close to working, my own cider-repl buffer that came from cider-connect-cljs to the shadow-cljs nrepl is stuck in a state pending-cljs (even though it is clearly sending code to the browser itself, this prevents cider-eval-* from working in clojurescript-mode buffers)

that, after putting in a global shadow-cljs.edn in ~/.shadow-cljs/config.edn with the most recent cider-nrepl version.

makes sense, that's something like what I did

I have had that pending-cljs message too.

https://github.com/clojure-emacs/cider/issues/2456

track-state middleware...

aha https://docs.cider.mx/cider/repl/configuration.html#repl-type-detection

"Also - are all of you using the latest piggieback?"

your pomeranian is gonna love this

🎉

try the following

eval this elisp (setq cider-repl-auto-detect-type nil)

then with your pending-cljs repl active

execute cider-set-repl-type and choose cljs

it is working for me

cider docs say this can happen "in rare cases such as a bug in shadow-cljs" (paraphrased)

doing it now

the reality is that whatever the track-state middleware is supposed to do to allow cider to detect that the repl has become a cljs repl is broken (perhaps the way biff is running shadow-cljs simply does not load the track-state middleware, or it could be that it loads it in the wrong order, etc)

but now I have cider+biff with cljs cider-eval-* and hot compilation of clojurescript (a la figwheel)

om#g.

:cider: ?

it works.

hooray

died. applying defibrillation paddles. back.

drink some cider

hahaha

I've dodged the Vim bullet, again.

and Don't Panic

well, I do use evil mode in emacs, but don't tell anyone.

a ton of people do... I used vim for 12 years and before I switched to emacs

you should make sure to write down how we got it working

This was epic. I am copying this entire session and having it engraved on a palladium plate.

I myself am a little tired and forgetful, though I think I know which bits were essential to my poking about

I'm keeping it all. Because next week I'll be right at the start of it again.

two palladium plates... one for launching beyond our solar system.

I'm sure they'll be able to puzzle out our entire planet by studying that.

hahaha

good lord, so grateful.

super excited to dive into the various parts of this framework now (crux, etc).

some cider features don't work...

TBH it's been a while since I clojurescripted and I am not sure how much of this is because it's using shadow-cljs ...

but the basics work nicely, which is what you wanted!

it's exactly what I wanted.

I use very few features (unfortunately)...

but here the ignorance is an asset.

haha, I'm glad I could help!

I am ridiculously glad you could.

😀

I am pretty sure I had cider-inspect working with cljs at some point and it is a nice feature, which it seems that the shadow-cljs nREPL server doesn't support even though the cider middleware is loaded

I'm always on the verge of giving up software altogether, then an awesome person like you or @dpsutton or some other kind soul will step in and get my boat pointed in the right direction again.

So thank you again.

Here is some more info, this is important: https://github.com/thheller/shadow-cljs/issues/62

essentially it says that the way shadow-cljs works this way at all (i.e. without piggieback) is by emulating (some of) piggieback

also relevant discussion here https://github.com/clojure-emacs/cider/issues/2239

oh @macrobartfast I just noticed one other potential snag for people trying to use biff with cider+nrepl ... I had a global ~/.nrepl/nrepl.edn file configuring the cider.nrepl/cider-middleware and this appears to break whatever auto-configuration shadow-cljs is doing when it detects that cider-nrepl is on the classpath (as described here: https://shadow-cljs.github.io/docs/UsersGuide.html#nREPL) That is, biff's shadow-cljs nREPL ONLY loads the cider middleware if I remove that ~/.nrepl/nrepl.edn file (which a lot of nrepl users will probably have lying around)

ok... so many gotchas.

I'm happy to remove gotchas when someone tells me about them. I don't use cider-nrepl personally so I'm not aware of any issues it may have with shadow-cljs unless someone tells me.

Totally... and no one could anticipate the issues from all the combinatorial possibilities of all the parts.

What's crazy to me is how often this all just works.

this is all from trying to use biff, which tries to helpfully start everything up and prevents us from putting the necessary dependencies in place. its a bit like lein-shadow where it's trying to be helpful and that means its not super helpful

I understand... my 'gotchas' was mainly about the whole process of getting biff to work in my setup...

hopefully when my knowledge increases I might be able to contribute to biff by helping to make a startup option that would enable easier use with Emacs... or at least that seemed like an option that would work...

for someone like me biff offers a ton of value in being an integrated working system... I can see how all the parts fit together.

not to mention speed of deployment of ideas I want to explore. But I understand the tradeoffs of any integrated system.