cljdoc

https://cljdoc.org/ & https://github.com/cljdoc/cljdoc
rschmukler 2020-02-12T18:08:15.024Z

When I go to manually trigger a rebuild on https://cljdoc.org/d/teknql/systemic/0.0.1-SNAPSHOT I get an exception. Is there anything I can do to rebuild?

martinklepsch 2020-02-13T13:54:27.027200Z

It seems that the code that determines pom and jar URLs doesn’t work. Specifically find-artifact-repository doesn’t properly identify Clojars as the repo. It seems that something changed in Clojars’ CDN so that the URLs changed from: https://repo.clojars.org/teknql/systemic/0.0.1-SNAPSHOT/ to https://repo.clojars.org/index.html?prefix=teknql/systemic/0.0.1-SNAPSHOT/ We’re doing a HEAD request checking if the first URL returns 200 OK but it now returns 301 Moved Permanently .

martinklepsch 2020-02-13T13:54:42.027400Z

Probably worth opening an issue on this if you don’t mind.

martinklepsch 2020-02-13T13:56:57.027600Z

@tcrawley is the URL change intentional and do you happen to know if all Maven repos support the new ?prefix style URLs?

2020-02-13T14:01:31.028Z

Yes, I moved the backend storage from Rackspace Cloudfiles to S3 over the weekend. Cloudfiles supported "directory" browsing, but S3 does not. So I used a tool that allowed you to browse by redirecting listings to a top level index w/js that generates an index. I didn't realize that would break things.

2020-02-13T14:01:51.028200Z

That's not something that other maven repos do as far as I know

2020-02-13T14:04:17.028500Z

I can give it some more thought and see if there is an alternate way to restore directory browsing if needed

martinklepsch 2020-02-13T14:07:08.028700Z

I’d be happy to special case Clojars if that’s whats needed but not quite seeing what specifically is necessary yet. Maybe curl -iL would be all that’s needed to detect if a thing exists.

martinklepsch 2020-02-13T14:07:49.028900Z

But chances are there’s also other ways to do what cljdoc is doing that are a little bit more robust.

2020-02-13T14:08:28.029100Z

What does clojars return if the path doesn't exist? If that still returns a 301 instead of a 404, I'd consider that a bug

martinklepsch 2020-02-13T14:08:32.029300Z

<https://github.com/cljdoc/cljdoc/blob/791a9a339b7aac18b41906b1dcd736af36a8547f/src/cljdoc/util/repositories.clj#L73>

martinklepsch 2020-02-13T14:08:58.029500Z

This is the current code that determines if a project/version combination is available at a specific repository

martinklepsch 2020-02-13T14:09:42.029700Z

It does return a 301 for projects/versions that don’t exist

martinklepsch 2020-02-13T14:10:11.030Z

eg. Does not exist:

curl -i <https://repo.clojars.org/teknql/systemic/0.0.3-SNAPSHOT/>
Exists:
curl -i <https://repo.clojars.org/teknql/systemic/0.0.1-SNAPSHOT/>

2020-02-13T14:12:10.030400Z

Ok, I think I can at least return a 404 in that case. I won't be able to look at it or give it more thought until this evening (US-Eastern), unfortunately

martinklepsch 2020-02-13T14:12:33.030600Z

No worries!

martinklepsch 2020-02-13T14:12:43.030800Z

It’s done when it’s done 🙂

martinklepsch 2020-02-13T14:12:51.031Z

Thanks for looking into it!

martinklepsch 2020-02-14T15:23:20.032900Z

Thanks very much @tcrawley, merged and deployed & it’s working 🙌

2020-02-14T15:24:03.033100Z

My pleasure! I'm glad that worked.

jimmy 2020-02-12T18:44:59.024200Z

There is a hidden rebuild button next to where it says scm info missing. I made it black in this screenshot but you should just barely be able to see it on hover.

rschmukler 2020-02-12T18:46:24.024700Z

Sorry! I found the button okay, but when I click it the page reports an exception

rschmukler 2020-02-12T18:46:39.024900Z

(No stack trace or anything is leaked, however)

jimmy 2020-02-12T18:47:52.025100Z

(I know nothing just knew about that button) Did you look at the section of the FAQ about scm issues? https://github.com/cljdoc/cljdoc/blob/master/doc/userguide/faq.md#how-do-i-set-scm-info-for-my-project

rschmukler 2020-02-12T18:48:49.025400Z

I think I got the SCM working - but to test it I need a rebuild... But if I click rebuild, it seems to cause cljdoc to hit an exception and crash. Presumably it's being logged somewhere

jimmy 2020-02-12T18:49:26.025600Z

You'd need to push a new version for it to pick up that information

rschmukler 2020-02-12T18:49:57.025800Z

For it to pick up new SCM info?

jimmy 2020-02-12T18:50:03.026Z

Maybe not a new version since it is a snapshot. But I really have no idea. Maybe someone more knowledgeable will have to reply.

rschmukler 2020-02-12T18:50:16.026200Z

Cool cool. Thanks for the help anyway 🙂