Clojurians Log v2

Alright guys, so here’s an issue I was having. I can write this JS and have it run in node no problem.

import unified from 'unified';
import parse from 'uniorg-parse';
import uniorg2rehype from 'uniorg-rehype';
import extractKeywords from 'uniorg-extract-keywords';
import html from 'rehype-stringify';
import { toVFile } from 'to-vfile';

unified()
    .use(parse)
    .use(extractKeywords)
    .use(uniorg2rehype)
    .use(html)
    .process(toVFile.readSync('src/posts/example.org'), function(err, file) {
        console.log(file);
    })

(ns app.parser
  (:require ["unified" :as unified]
            ["uniorg-parse" :as parse]
            ["uniorg-extract-keywords" :as extract-keywords]
            ["uniorg-rehype" :as uniorg2rehype]
            ["rehype-stringify" :as html]
            ["fs" :as fs]
            ["to-vfile" :refer (toVFile)]
            [cljs-node-io.core :as io]))

(->
  (unified)
  (.use parse)
  (.use extract-keywords)
  (.use uniorg2rehype)
  (.use html)
  (.process (.readSync toVFile "src/posts/example.org")
            (fn [err file]
              (js/console.log file))))

app.parser => (->
(unified)
(.use parse)
(.use extract-keywords)
(.use uniorg2rehype)
(.use html)
(.process (.readSync toVFile "src/posts/example.org")
(fn [err file]
  (js/console.log file))))

Execution error (TypeError) at (<cljs repl>:1).
Cannot read property 'readSync' of undefined
:repl/exception!

app.parser => (->
(unified)
(.use parse)
(.use extract-keywords)
(.use uniorg2rehype)
(.use html)
(.process (toVFile "src/posts/example.org")
(fn [err file]
  (js/console.log file))))

Execution error (TypeError) at (<cljs repl>:1).
shadow.js.shim.module$to_vfile.toVFile is not a function
:repl/exception!

don't know. what setup is this?

node-script build? node-repl?

I’m using node-repl with cider.

and what do you get for (require '["to-vfile" :as x]) and x?

Let me check

and (.. x (-toVFile) (readSync "src/posts/example.org"))

app.parser> (require '["to-vfile" :as x])
nil
app.parser> x
#js {}

app.parser> (.. x (-toVFile) (readSync "src/posts/example.org"))

Execution error (TypeError) at (<cljs repl>:1).
shadow.js.shim.module$to_vfile._toVFile is not a function
:repl/exception!

and what do you get for (js/require "to-vfile")?

app.parser> (js/require "to-vfile")

Execution error (Error) at (<cljs repl>:1).
Must use import to load ES Module: /Users/main/Code/web/mysite/node_modules/to-vfile/index.js
require() of ES modules is not supported.
require() of /Users/main/Code/web/mysite/node_modules/to-vfile/index.js from /Users/main/Code/web/mysite/[stdin] is an ES module file as it is a .js file whose nearest parent package.json contains "type": "module" which defines all .js files in that package scope as ES modules.
Instead rename index.js to end in .cjs, change the requiring code to use import(), or remove "type": "module" from /Users/main/Code/web/mysite/node_modules/to-vfile/package.json.

:repl/exception!

It’s so odd, because toVFile is so clearly a function.

well the error you just posted is probably your issue

Does this mean I need to fix the package myself?

the package probably is not your problem

do you have type: module in YOUR package.json?

No I don’t.

hmm I'm unsure what the current state of things is in node

shadow-cljs uses require to load npm stuff, can't remember if there was something you are supposed to set to enable that

> Install > This package is https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c: Node 12+ is needed to use it and it must be imported instead of required. From the https://github.com/vfile/to-vfile page for to-vfile.

(also likely depends on which node version you use since this stuff changed a lot)

I guess this is a problem. If it cannot be brought in using require then I need to either do a workaround or help fix shadow to add support for this.

yeah it is a topic I have been putting off .. basically just waiting for the node world to sort this stuff out

they have been going back and forth for years on how to do this, guess they finally settled

not sure if there is an easy fix, need to read up on how node actually handles this now

I see. How would you recommend a novice start to chip away at understanding a large codebase? I’d like to see this implemented, so it would be well worth my time to attempt to fix it.

well it'll be something along :target :esm https://clojureverse.org/t/generating-es-modules-browser-deno/6116

since this really only works when everything is ESM, including the code shadow-cljs generates

it might even work today, just may require some additional options

:target :esm currently defaults to trying to bundle all dependencies, but there is :js-options {:keep-as-import #{"fs" "to-vfile" ...}} for stuff it shouldn't bundle

although bundling may actually be fine too, not sure what kind of code this is

this will really involve a lot of rewriting in the node interop code, maybe even a new seperate implementation so we don't lose the old commonjs stuff

wonder why you don't get the same ESM error for (require '["to-vfile" :as x]), that just does a js/require under the hood

I imagine error handling can be pretty messy.

Hello! How to troubleshoot

207 |     (def ui-player (comp/factory Player {:keyfn :player/id}))
-----------^--------------------------------------------------------------------
 ui-player at line 207 is being replaced

Sometimes when I have weird errors, stopping everything and removing compiler cache (https://shadow-cljs.github.io/docs/UsersGuide.html#_compiler_cache) helps. Other times I am just removing or commenting out code piece by piece until it starts working.

Thanks! Restarting helped.

What’s the best way to include a node script in a shadow project? Where might I put it in my project? I thought of using a lib folder just for parser.js. All I want really is to call the functions in the script from cljs, that way I can also debug the script somewhat interactively.

.
├── LICENSE
├── README.md
├── deps.edn
├── package.json
├── postcss.config.js
├── shadow-cljs.edn
├── src
│   ├── app
│   │   ├── components.cljs
│   │   ├── core.cljs
│   │   ├── data.cljs
│   │   ├── index.clj
│   │   └── parser.js
│   ├── config
│   │   └── site-data.edn
│   ├── css
│   │   ├── 404.css
│   │   ├── main.css
│   │   └── tailwind.css
│   ├── lib
│   └── posts
│       ├── <http://001-first-post.org|001-first-post.org>
│       ├── <http://002-parsing-org-docs.org|002-parsing-org-docs.org>
│       ├── <http://100-test.org|100-test.org>
│       ├── blog-posts.edn
│       └── <http://example.org|example.org>
├── tailwind.config.js
├── yarn-error.log
└── yarn.lock

@c.westrom see https://shadow-cljs.github.io/docs/UsersGuide.html#classpath-js