ring-swagger & compojure-api
Empperi 2019-02-05T09:23:06.064Z

Trying to get compojure-api + aleph + core.async combo to work. Unfortunately my channels end up into compojure render phase and there cause an exception

Empperi 2019-02-05T09:23:14.064200Z

Trying to figure out which part am I missing

Empperi 2019-02-05T09:23:38.064600Z

I have required compojure.api.async so that it extends the necessary protocols

Empperi 2019-02-05T09:24:37.065200Z

https://github.com/metosin/compojure-api/wiki/Async says that aleph needs manual bootstrapping. Is this still the case? This page has been updated in 2017

Empperi 2019-02-05T09:26:10.065400Z

And if so, what that would actually mean?

Empperi 2019-02-05T09:26:24.065800Z

I'm having hard time to understand what could possibly be needed

Empperi 2019-02-05T09:26:50.066300Z

I mean, I have a fully working aleph server running, everything else works except async stuff

Empperi 2019-02-05T09:27:31.067Z

I would prefer using core.async since I need to do relatively complex concurrent querying which then will translate into the actual response which I'll return from my API

Empperi 2019-02-05T09:28:31.067500Z

Guess I can just read the channel in a blocking manner just before I return something from my handlers but that sounds so lame...

Empperi 2019-02-05T13:54:54.070900Z

The more I've dug into this the harder this problem seems to be. Now I've dumped Aleph and taken Jetty. Now I manage to serialize channels but unfortunately not via compojure-api but via my own system. However, what I've realized here is that it isn't strictly obvious how one should serialize the channel. For example, the main reason why channels are nice as response medium is that they can be written "as they come". So when it comes to JSON one should have one put operation per JSON array element. However, what if one wants to send just a JSON object without array wrapper? Should we just write an object if all we get from the channel is just one element? The answer is of course "no" since in that case we couldn't have one element long arrays which are definitely needed.

Empperi 2019-02-05T13:56:37.071600Z

Yeah, I've seen that

ikitommi 2019-02-05T13:57:14.071900Z

have you checked https://github.com/metosin/compojure-api/tree/master/examples/async

Empperi 2019-02-05T13:57:18.072200Z


Empperi 2019-02-05T13:57:53.072500Z

Also, do you have an idea regarding the problem I described above? ๐Ÿ™‚

Empperi 2019-02-05T13:58:06.072800Z

Basically sounds to me like an unsolvable problem without some additional information

ikitommi 2019-02-05T13:58:53.073600Z

oh, you would also need a streaming json writer

Empperi 2019-02-05T13:58:55.073700Z

One could use spec definitions to deduce if the result should be an array or not

ikitommi 2019-02-05T13:59:33.074200Z

do you have large data sets you need to return?

Empperi 2019-02-05T13:59:50.074600Z

Yes and no. No right now, yes in the future

Empperi 2019-02-05T14:00:51.075Z

So I could just ignore it at this point

Empperi 2019-02-05T14:00:57.075300Z

But that feels... Dirty ๐Ÿ™‚

ikitommi 2019-02-05T14:01:23.076300Z

if you need to stream data, I would use sse or web sockets and push separate full json data blobs out

Empperi 2019-02-05T14:01:35.076800Z

And basically if I did that then the easiest solution would be just to consume the whole thing at handler level, do the serialization there etc since that's where I know what kind of structure the data should have

ikitommi 2019-02-05T14:02:10.077500Z

writing a large json-array in parts would require to write the json internally in parts too.

ikitommi 2019-02-05T14:02:30.078200Z

if the data is not big, just collect the bits via core.async and when itโ€™s done, flush it out?

Empperi 2019-02-05T14:03:08.079Z

Yeah, that is practically the only thing I can think of which would make this in any way reasonable

Empperi 2019-02-05T14:03:14.079200Z

Without excessive amount of work

Empperi 2019-02-05T14:03:35.080Z

Practically need to skip all the nice automatic conversions of compojure-api then though

Empperi 2019-02-05T14:03:50.080400Z

or wait. Not really

ikitommi 2019-02-05T14:03:53.080700Z

yeah, you canโ€™t do things like validate the output if you stream out the data in parts etc.

Empperi 2019-02-05T14:04:52.081700Z

Right. Need to push this to higher abstraction level and do the work there

ikitommi 2019-02-05T14:05:06.082100Z

but, returning a stream of json parts is also a different contract to a returning a full data

ikitommi 2019-02-05T14:05:35.082600Z

you can still validate the parts manually in the handler, if you would stream it out.

Empperi 2019-02-05T14:05:45.083Z


Empperi 2019-02-05T14:06:12.083600Z

I have streaming use case right from the bat but fortunately it's a bit of a special case where I do not need to care about this stuff

Empperi 2019-02-05T14:06:33.084100Z

Cheers. Needed some brain racking.

Empperi 2019-02-05T14:06:40.084300Z

(flu, not at my best right now)


hey @ikitommi do you have any pointers on this? Something I could investigate

ikitommi 2019-02-05T17:41:22.084900Z

I would try to run the coercion directly with spec-tools to isolate all c-api things. There is st/decode (runs both coerce and conform), st/coerce (best-effort coercion with form walking) and st/conform coercion via conforming. Sorry, really busy with a client project, hopefully time on Friday to help with all the piled issues. Ping @mping


No worries, I appreciate your efforts


I'll dive and see if I can spot anything shady