clojure-uk

A place for people in the UK, near the UK, visiting the UK, planning to visit the UK or just vaguely interested to randomly chat about things (often vi and emacs, occasionally clojure). More general the #ldnclj
djm 2021-05-20T05:09:35.043700Z

πŸ‘‹

dharrigan 2021-05-20T06:20:08.043900Z

Maidin mhaith

djm 2021-05-20T06:28:01.045400Z

I see that freenode is imploding. It had been a while since I was in #clojure on there (it was pretty quiet when I was last in). But looks like at least some people have moved over to libera now.

djm 2021-05-20T06:28:40.046Z

(Annoying that Slack won't let you type #foo without it linking to #foo, if that exists as a channel)

dharrigan 2021-05-20T06:29:31.046300Z

a netsplit

dharrigan 2021-05-20T07:29:44.046600Z

I'm aware, just making a somewhat-related injoke

dharrigan 2021-05-20T07:29:58.046800Z

however, I'm on Freenode, and I'm not seeing a mass exodus in the channels I'm in

djm 2021-05-20T07:30:50.047Z

I'm not in many channels on freenode anymore anyway, and some of those that I'm in are fairly moribund these days

djm 2021-05-20T06:36:30.046400Z

The downside to text-based communication - I can't tell if you're aware of what's going on, and that's a joke, or if you're unaware and are referring to a literal netsplit

Aron 2021-05-20T08:17:20.047300Z

morning

Aron 2021-05-20T08:17:41.047700Z

freenode imploding? to me it seemed like a successful hostile takeover

djm 2021-05-20T08:20:04.047800Z

Sure, followed by lots of staff and at least some users leaving.

djm 2021-05-20T08:20:51.048Z

It seemed like a lot of staff. No idea how many staff freenode has altogether these days.

Aron 2021-05-20T08:21:06.048200Z

I see what you mean

thomas 2021-05-20T09:22:04.048500Z

morning

Luis Thiam-Nye 2021-05-20T10:44:04.048700Z

Good Morningβ„’

2021-05-20T11:03:24.048900Z

Morn'

alexlynham 2021-05-20T11:36:25.049100Z

morning

2021-05-20T17:34:47.051700Z

Hey, are there any tutorials or anything that really walk someone through core.async or promises in clojurescript? I'm really trying to understand what's going on here and it just ends up being a confusing mess. It's really quite frustrating because I can write async code in python et al really easily, but doing so in clojurescript just doesn't seem to click at all. I invariably get a channel or a promise and I can't get the value out of it... Actually is there some way to drive it in the REPL better? Part of my frustration is that it's very hard to tell if things are working at the REPL. I just get back a ManyToManyChannel, so I feel like I have very little visibility into how it's working.

mccraigmccraig 2021-05-20T18:27:55.053200Z

what constructs would you use in python @folcon? not promises ?

2021-05-20T18:28:12.053700Z

primarily multiprocessing

mccraigmccraig 2021-05-20T18:32:30.057900Z

if i need to inspect something in the cljs repl i do something like (defn pval [p] (let [a (atom ::undefined)] (p/then p #(reset! a %)) a))

2021-05-20T18:32:53.058600Z

I think my primary issue is not being clear what I'm working against and the different constructs that I've seen give me inconsistent results. For example, Using promises, I'm currently reading in a zip file and walking through the entries. I console.log the entry name and contents and they don't match. The entry name changes, but the contents seem to be permanently pointing to the first entry. Not really sure what's going on there. I'd prefer to use core.async, but I'm not sure how to work with it in a repl in clojurescript. As I mentioned previously I constantly get ManyToManyChannel being the evaluation result, so how do you evaluate functions and get the output to test your assumptions?

mccraigmccraig 2021-05-20T18:32:53.058700Z

(since there's no deref available for js promises)

2021-05-20T18:32:59.058900Z

Hmm

2021-05-20T18:33:39.059200Z

Then bind a to some def and poke it?

mccraigmccraig 2021-05-20T18:35:02.059700Z

yep... same idea works for core.async too

mccraigmccraig 2021-05-20T18:35:49.060600Z

although unless you are talking promise-chans then chans are going to behave differently

mccraigmccraig 2021-05-20T18:43:05.061600Z

if you are wanting to test the async outputs in unit-tests then async is your friend - https://clojurescript.org/tools/testing#async-testing

2021-05-20T18:44:50.061900Z

I've not even gotten that far yet πŸ˜ƒ

mccraigmccraig 2021-05-20T18:47:04.062300Z

what sort of async ops are you doing ?

2021-05-20T18:57:58.062600Z

broadly this: https://github.com/transcend-io/conflux/issues/63

2021-05-20T18:58:56.063100Z

Got a bunch of xml files inside a zip I'm trying to read

mccraigmccraig 2021-05-20T19:03:32.064Z

so you are having to convert from a js promises/streams API into something cljs ?

2021-05-20T19:03:53.064400Z

Getting lost trying to work with this and then mixing in streams and file api's stuff not working and having to convert to blobs etc...

2021-05-20T19:04:04.064700Z

well that would be the ideal outcome

2021-05-20T19:04:14.065Z

I'm trying to read the file in chunks if at all possible

2021-05-20T19:05:38.066600Z

I've gotten it working in the past where I get a massive string for the file contents, but that's really unhelpful when someone loads a giant file

mccraigmccraig 2021-05-20T19:09:52.067800Z

it sounds entirely doable, but the mix of promises and streams in the js api probably presents some challenges mapping it over to cljs

mccraigmccraig 2021-05-20T19:12:05.068700Z

we do loads of stuff like that on the backend, where we use manifold for both promises+streams, but i don't think we do anything like it in js

2021-05-20T19:16:13.069600Z

I'm pretty sure it's doable, just currently not clear how to attack it πŸ˜ƒ...

mccraigmccraig 2021-05-20T19:22:14.072400Z

my default line of attack would be to use promesa for promises, write something to convert between js streams and core.async chans, and write something which reduces a core.async chan to a promise - that would give you something which looks roughly like the manifold api

2021-05-20T19:35:08.073300Z

Thanks for the tip though πŸ˜ƒ... I'll try it out!

mccraigmccraig 2021-05-20T19:45:58.074400Z

oh, i hadn't seen <p! , that looks ok too

2021-05-20T19:52:23.075100Z

Are there any good resources around working with streams in clojurescript?

2021-05-20T19:52:49.075600Z

I'm writing a lot of ad-hoc code and it feels very inelegant πŸ˜ƒ...

mccraigmccraig 2021-05-20T20:38:36.075700Z

don't know - I've never worked with js streams directly... but if you convert pipe then to/from core.async chans then you can just work with the chans

2021-05-20T20:47:28.075900Z

Thanks anyway πŸ˜ƒ

2021-05-20T23:44:10.076400Z

Finally figured it out, you're supposed to request a promise nested within the entry promise >_<...

mccraigmccraig 2021-05-21T07:54:25.077100Z

huh ?

2021-05-21T13:14:47.078400Z

The zip lib, I couldn't work out where the data was, I kept reading the buffer expecting it to contain the data, instead you have to within the promise call a function that doesn't appear in the method list (had to read source to find it) which creates another promise which when it resolves gives the file contents...

mccraigmccraig 2021-05-21T13:44:01.078800Z

oh, right, so there's a first "open the zipfile" step, then repeated "fetch" steps to get at the actual data ?

2021-05-21T14:41:43.079Z

It looks like opening the zipfile is more opening a table of contents? Then yes, fetching the contents... One thing I'm uncertain about is that a core async chan is supposed to have a buffer limit right? Is that going to be a problem for this kind of workflow where I could have a zip file with lots of entries and I can't queue them all up?

mccraigmccraig 2021-05-21T14:46:45.079300Z

yeah, i think zip/tar are like a filesystem inside a file, with index/metadata pointing off to other offsets inside the file

mccraigmccraig 2021-05-21T14:50:41.079500Z

one of the nice things about core.async (and also manifold and rx and js streams and vert.x etc) is backpressure - and core.async handles it very nicely. your &gt;! operation will not "return" until buffer space is available, so your producer process will be "parked" until buffer space is available... symmetrically, your consumer process will park on &lt;! until something is available

mccraigmccraig 2021-05-21T14:52:52.079700Z

i use quotes because that's sync terminology, and it's all really async - the go macro transforms everything into callbacks, so &gt;! gets transformed into a put! with a callback and &lt;! gets transformed into a get! with a callback etc

mccraigmccraig 2021-05-21T14:55:19.079900Z

but in summary, you don't really have to worry about filling buffers up - there aren't any buffers by default, and your produce will only go at the pace your consumers dictate. if you add some buffers in between producer and consumer then that lets your consumers lag a bit without stopping your producer. in this particular case, where you have control of both producer (whatever is parsing the zipfile and putting content onto a chan) and consumer (whatever is reading content from the chan) you may not need any buffers at all

2021-05-21T15:27:03.080100Z

Does that still work in a js scenario?

mccraigmccraig 2021-05-21T15:31:05.080300Z

yes, absolutely - it's not blocking, it's parking - it makes a continuation closure fn, which is registered as a callback, to be called when data is available

mccraigmccraig 2021-05-21T15:32:58.080500Z

not a million miles from what js async/await does, or what the promesa/let macro does

2021-05-22T19:25:09.080800Z

Ok that's very cool, going to have to try that out tomorrow...