Clojurians Log v2

looks like things haven't changed so much w/ React, 5 dynamically referred to plugins still need to be in externs when advanced compiling

still same codebase as the above the following funnels everything through Closure instead

{:main cljs-bundle.core
 :output-to "out/main.js"
 :output-dir "out"
 :language-out :es6
 :npm-deps true
 :externs ["externs.js"]}

browser REPL quick start works like a charm with the bundle changes

Not sure if that’s us - we use shadow for npm stuff and it also handles code-splitting. Kind of a necessity when eg we use Vega which is 1mb of JS.

Is optimal code-splitting different than ordinary code splitting? Is this the thing that shadow-cljs does?

Excuse my ignorance, I am only exposed to the way shadow-cljs does things :)

hey let's not use threads, it's really annoying to see backlog

optimal code splitting is a Google Closure thing - not a CLJS or shadow-cljs thing

neither tool can do what Closure does for Closure-aware code

all you can do for npm stuff is prepend it

but now that I think about it, the existing code-splitting feature probably already works or can be made to work trivially

that said if we can't - not a big deal IMO, if shadow-cljs can offer this feature and we can't it doesn't really have any bearing on the bigger goal

code-splitting is a great feature - but it doesn't have anything to do really w/ the ecosystem effects I'm interested in

the issue with code-splitting is getting that to work in webpack

because that has a fundamentally different view on how to do that then the closure-compiler does

FWIW in shadow-cljs I have a :external js-provider which looks pretty similar to the new bundle stuff (ie. generate a .js file that other tools can process to load npm deps)

I only did that as an experiment though so nobody uses it

> the ecosystem effects I'm interested in

what would that be?

you don't need CLJSJS like solutions to distribute ClojureScript libraries that depend on the JS ecosystem

that's the whole point of doing the bundle thing - not convenience

nor is it about tool X vs. X do CLJS builds - just don't care about that at all

the important thing is that any tool X can handle a CLJS dep that uses :npm-deps to rely on something

@thheller re: code-splitting, how do you handle requires that you can't process i.e. React components w/ image / CSS requires?

just ignore them. they aren't too common in actual published libs (for the browser)

so users run another tool to handle them?

they are common in react-native but shadow-cljs doesn't process that at all

so you can't for example use shadow code splitting to partition assets based on what components load on a page?

it can in theory but it doesn't no

but most libraries actually do not have css requires so not a big deal

re: webpack yeah not interested in trying to combine w/ their code splitting - it can't work

however the module grouping algorithm doesn't work on source, it only works on dependencies - so I think it can probably be made to work pretty easily

while processing node stuff through Closure is a hit or miss - our ability to compute the dep graph is pretty good

so we can just add these to the module grouping algorithm and mark them as coming from elsewhere

then webpack is just used to prepend that stuff

just as in the single file case

for the :external case in shadow-cljs I intended that file to be loaded completely separately which effectively is sort of code-splitting out the npm code and the cljs code

can't actually split npm deps but its usually good enough if you can separate out the npm parts and have that cacheable

yeah that's probably what we'll ship w/ now

and add the code splitting part as a separate step

it would be nice if all Clojure dep tools could fetch npm deps rather than ClojureScript caring about it, but I don't see that happening quickly in a synchronized fashion, @alexmiller thoughts?

unlike git deps it's not meaningful if only tools-deps can do it - every tool needs to be able to do it a la Maven

if you intend to use webpack you'll have npm so just using npm seems fine to me

I played with the idea of replacing it myself but its just not worth

there are already alternatives like yarn anyways

@thheller of course it has to bottom out in npm

I'm just pointing there's no a universal Clojure way to have a node dep - curious if Alex has thoughts is all

ClojureScript being able to handle is a acceptable workaround for now - just looking at the much longer picture too

I have actually thought about this a bit

tools.deps is designed to procure and expand transitive deps through an abstraction (a few multimethods) and npm has everything needed to fulfill that

@alexmiller yeah that's why I was asking, you mentioned it before

The question then is - what do you do with that? How does that relate to the classpath?

it doesn't need to relate to classpath

all that stuff just ends up in node_modules

something else will deal with it

my question is really more about whether you thought other tooling

since if it works w/ tool-deps not much of a win

not useful for Lein / Boot etc. w/o a lot more steps

So how would you get at this behavior though? Programmatically by calling into tools.deps or do you want it to work via clj?

my expectation would be that just installs that stuff

to clarify we don't need classpath because node is just relative requires starting at node_modules

any build tool that is going to use that stuff understands that already

Only in the project or in a shared place like m2?

I don't think we want to go there

yarn and npm have slightly different semantics

so local only

I’m not suggesting anything, I just don’t know how any of this stuff works :)

This is a major deviation from a model Rich and I have worked pretty hard to nail down so would need some real thought about whether this is the best place to do this (I’m a little worried to back into it by “this things looks kinda like that thing”)

We have also been working on this tools.build stuff for artifact building and that may be a better fit, but we are still trying to figure out the shape of it

right, ok, just wanted to your thoughts

there's no issue IMO w/ us handling it - but wanted to get your initial thoughts before we release

@alexmiller are site builds automatic again?

@dnolen no, ping me when needed

@mfikes hrm, I guess because of spec.alpha and reader stuff we actually need Clojure 1.9? Or am I misremembering?

if that's true I'm not sure why we didn't bump this in the pom.template.xml?

Hmm... I can't recall

trying with Clojure 1.8

Clojure 1.8 works fine

@alexmiller kick it off whenever you have a chance

I removed a lot of stuff and modernized everything - clj and cljs.main only in the guides

@mfikes no rush but if you want to put together release notes that would be cool - I'm going to work on a post about :target bundle + updated Webpack guide

site is updated

thanks!

https://github.com/clojure/clojurescript-site/blob/april-release-target-bundle/content/guides/webpack.adoc

if people want to try it out

for the ClojureScript dependency you'll either want to ./script/build and note the version that gets install locally

or git deps

{:deps {org.clojure/clojurescript {:git/url "<https://github.com/clojure/clojurescript.git>" :sha "7792adf10e8961cbd5b0267a30a7d9655c770086"}}}

@dnolen 1. npm --save-dev webpack webpack-cli needs install 2. same for the npm --save react react-dom (isn't --save implied by install?) 3. the clj -m cljs.main -co build.edn -v -c -s assumes a deps.edn file has been created already but that's not in the steps 4. going through these steps i got

Execution error (ExceptionInfo) at cljs.closure/build (closure.clj:3192).
:bundle-cmd :none failed

:std-out
  18   │      "Hash: 980d631d7dd0f2217840\nVersion: webpack 4.42.1\nTime: 24ms\nBuilt at: 04/12/2020 2:56:09 PM\n\nERROR in Entry m
       │ odule not found: Error: Can't resolve './out/index.js' in '/Users/dan/projects/clojure/scratch/hello-bundler'\n"}
...
   :cause ":bundle-cmd :none failed",
  47   │   :data
  48   │   {:cmd ["npx" "webpack" "--mode=development"],
  49   │    :exit-code 2,
  50   │    :std-out

@dpsutton yeah I'm going through it now - updated

in Firefox i'm getting > SyntaxError: import declarations may only appear at top level of a module and no console logging

@dpsutton did you try Chrome/Safari?

Safari reported invalid { This machine is only two days old and doesn’t have chrome yet

> import {npmDeps} from "./npm_deps.js"; seems to be the offending line. Is this the correct output of webpack?

no

@dpsutton I think you missed a step

I just copied and pasted everything and it worked (except the final step, advanced compilation + foreign override)

your issue not browser related - but something else - make sure you copy and paste unless you really understand the steps, no tweaks (happy to explain)

Yeah tracing through to see which one I may have overlooked

:output-to must be out/index.js

entry: in webpack config must be out/index.js

and the final file must be out/main.js to see it in the default webserver

all of this can of course be changed - but these things must align

@dpsutton you can also drop :build-cmd and run webpack directly - that part is pure convenience

ok i think i had missed installing webpack and webpack-cli when making my list earlier. sorry about that

now everything is working in the browser. however, in the terminal i'm seeing

Exception in thread "Thread-386" java.lang.NullPointerException
	at clojure.core$deref_future.invokeStatic(core.clj:2300)
	at clojure.core$deref.invokeStatic(core.clj:2320)
	at clojure.core$deref.invoke(core.clj:2306)
	at cljs.repl.browser$repl_client_js.invokeStatic(browser.clj:116)
	at cljs.repl.browser$repl_client_js.invoke(browser.clj:115)
	at cljs.repl.browser$send_repl_client_page.invokeStatic(browser.clj:123)
	at cljs.repl.browser$send_repl_client_page.invoke(browser.clj:118)
	at cljs.repl.server$dispatch_request.invokeStatic(server.clj:192)
	at cljs.repl.server$dispatch_request.invoke(server.clj:182)
	at cljs.repl.server$handle_connection.invokeStatic(server.clj:206)
	at cljs.repl.server$handle_connection.invoke(server.clj:202)
	at cljs.repl.server$server_loop$fn__9753.invoke(server.clj:216)
	at clojure.core$binding_conveyor_fn$fn__5754.invoke(core.clj:2030)
	at clojure.lang.AFn.run(AFn.java:22)
	at java.base/java.lang.Thread.run(Thread.java:830)

both safari and FF are making a request to http://localhost:9000/repl?xpc=%7B%22cn%22%3A%22qMRTfKzE3R%22%2C%22tp%22%3Anull%2C%22osh%22%3Anull%2C%22ppu%22%3A%22http%3A%2F%2Flocalhost%3A9000%2Frobots.txt%22%2C%22lpu%22%3A%22http%3A%2F%2Flocalhost%3A9000%2Frobots.txt%22%7D and that is 500 ing

http://localhost:9000/repl?xpc={"cn":"qMRTfKzE3R","tp":null,"osh":null,"ppu":"http://localhost:9000/robots.txt","lpu":"http://localhost:9000/robots.txt"}

@dpsutton yeah that's side detail, will look into that later

glad it's working!

this is awesome. thanks so much!

there's definitely an issue w/ inferred externs, which I need to dig into

making sure that Reagent can work is good test case

@dpsutton yeah it's pretty cool, to be honest we could have done it a long time ago, now that I really see it, a variant of this had been suggested a few times by folks like @thheller - like just emitting require for CLJS namespaces so you can run JS tools

one problem is JS tools can't really handle CLJS well so the impact seemed minor

other issues are how to make this work for REPLs w/o being a pain in the butt - re-natal actually gave me the idea here - perhaps shadow does something similar don't know

anyway I think this is a pretty big win

since what this gives you is dead simple

we just generate stuff that you can pass onto your favorite JS tool

I will add this bundle target with webpack as a test environment to Reagent repository

I'll also remove cljsjs dependency in the next version, and update documentation to instruct how to provide React.

@juhoteperi cool! I do need to sort out the inferred externs issue first - somehow COMPILED is sneaking in there when trying this w/ Reagent

Looks like bundle-cmd isn't run for me automatically, but it works when I run webpack myself.

@juhoteperi it should work, it might have failed though it's strange there's no error for you

(though that could be a bug)

@juhoteperi https://github.com/clojure/clojurescript/commit/66f6bc53b6a80a6cc6bb5049934f91fbd5148e8f

could be consolidated too

Couldn't find out reason. No error, no output and no file being written. Got the test suite running now, I passed the bundle to Karma with Webpack plugin, just needed a small workaround to get Karma to wait untill JS files loaded by Cljs are ready: https://github.com/reagent-project/reagent/pull/486/files

@juhoteperi do you have --ignore-scripts set for npm by chance? it’s npm option, but can be stored also in global config

ok I found the problem with advanced compilation - reagent project with react swapped in via webpack works fine under advanced compilation

@juhoteperi you should be able to reproduce on your REPL on your machine

(require '[clojure.java.shell :as sh]) (sh/apply ...) whatever your bundle command vector is

fwiw, I did get a case where the cmd failed for me and got an expected exception from the bad exit code