cljdoc

https://cljdoc.org/ & https://github.com/cljdoc/cljdoc
martinklepsch 2020-02-13T13:54:27.027200Z

It seems that the code that determines pom and jar URLs doesn’t work. Specifically find-artifact-repository doesn’t properly identify Clojars as the repo. It seems that something changed in Clojars’ CDN so that the URLs changed from: https://repo.clojars.org/teknql/systemic/0.0.1-SNAPSHOT/ to https://repo.clojars.org/index.html?prefix=teknql/systemic/0.0.1-SNAPSHOT/ We’re doing a HEAD request checking if the first URL returns 200 OK but it now returns 301 Moved Permanently .

martinklepsch 2020-02-13T13:54:42.027400Z

Probably worth opening an issue on this if you don’t mind.

martinklepsch 2020-02-13T13:56:57.027600Z

@tcrawley is the URL change intentional and do you happen to know if all Maven repos support the new ?prefix style URLs?

2020-02-13T14:01:31.028Z

Yes, I moved the backend storage from Rackspace Cloudfiles to S3 over the weekend. Cloudfiles supported "directory" browsing, but S3 does not. So I used a tool that allowed you to browse by redirecting listings to a top level index w/js that generates an index. I didn't realize that would break things.

2020-02-13T14:01:51.028200Z

That's not something that other maven repos do as far as I know

2020-02-13T14:04:17.028500Z

I can give it some more thought and see if there is an alternate way to restore directory browsing if needed

martinklepsch 2020-02-13T14:07:08.028700Z

I’d be happy to special case Clojars if that’s whats needed but not quite seeing what specifically is necessary yet. Maybe curl -iL would be all that’s needed to detect if a thing exists.

martinklepsch 2020-02-13T14:07:49.028900Z

But chances are there’s also other ways to do what cljdoc is doing that are a little bit more robust.

2020-02-13T14:08:28.029100Z

What does clojars return if the path doesn't exist? If that still returns a 301 instead of a 404, I'd consider that a bug

martinklepsch 2020-02-13T14:08:32.029300Z

<https://github.com/cljdoc/cljdoc/blob/791a9a339b7aac18b41906b1dcd736af36a8547f/src/cljdoc/util/repositories.clj#L73>

martinklepsch 2020-02-13T14:08:58.029500Z

This is the current code that determines if a project/version combination is available at a specific repository

martinklepsch 2020-02-13T14:09:42.029700Z

It does return a 301 for projects/versions that don’t exist

martinklepsch 2020-02-13T14:10:11.030Z

eg. Does not exist:

curl -i <https://repo.clojars.org/teknql/systemic/0.0.3-SNAPSHOT/>
Exists:
curl -i <https://repo.clojars.org/teknql/systemic/0.0.1-SNAPSHOT/>

2020-02-13T14:12:10.030400Z

Ok, I think I can at least return a 404 in that case. I won't be able to look at it or give it more thought until this evening (US-Eastern), unfortunately

martinklepsch 2020-02-13T14:12:33.030600Z

No worries!

martinklepsch 2020-02-13T14:12:43.030800Z

It’s done when it’s done 🙂

martinklepsch 2020-02-13T14:12:51.031Z

Thanks for looking into it!

seancorfield 2020-02-13T21:40:01.031800Z

I just ran into the build exception as well and was about to report until I saw that thread @tcrawley 🙂