When I go to manually trigger a rebuild on https://cljdoc.org/d/teknql/systemic/0.0.1-SNAPSHOT I get an exception. Is there anything I can do to rebuild?
It seems that the code that determines pom
and jar
URLs doesn’t work. Specifically find-artifact-repository
doesn’t properly identify Clojars as the repo. It seems that something changed in Clojars’ CDN so that the URLs changed from:
https://repo.clojars.org/teknql/systemic/0.0.1-SNAPSHOT/
to
https://repo.clojars.org/index.html?prefix=teknql/systemic/0.0.1-SNAPSHOT/
We’re doing a HEAD request checking if the first URL returns 200 OK
but it now returns 301 Moved Permanently
.
Probably worth opening an issue on this if you don’t mind.
@tcrawley is the URL change intentional and do you happen to know if all Maven repos support the new ?prefix
style URLs?
Yes, I moved the backend storage from Rackspace Cloudfiles to S3 over the weekend. Cloudfiles supported "directory" browsing, but S3 does not. So I used a tool that allowed you to browse by redirecting listings to a top level index w/js that generates an index. I didn't realize that would break things.
That's not something that other maven repos do as far as I know
I can give it some more thought and see if there is an alternate way to restore directory browsing if needed
I’d be happy to special case Clojars if that’s whats needed but not quite seeing what specifically is necessary yet. Maybe curl -iL
would be all that’s needed to detect if a thing exists.
But chances are there’s also other ways to do what cljdoc is doing that are a little bit more robust.
What does clojars return if the path doesn't exist? If that still returns a 301 instead of a 404, I'd consider that a bug
<https://github.com/cljdoc/cljdoc/blob/791a9a339b7aac18b41906b1dcd736af36a8547f/src/cljdoc/util/repositories.clj#L73>
This is the current code that determines if a project/version combination is available at a specific repository
It does return a 301 for projects/versions that don’t exist
eg. Does not exist:
curl -i <https://repo.clojars.org/teknql/systemic/0.0.3-SNAPSHOT/>
Exists:
curl -i <https://repo.clojars.org/teknql/systemic/0.0.1-SNAPSHOT/>
Ok, I think I can at least return a 404 in that case. I won't be able to look at it or give it more thought until this evening (US-Eastern), unfortunately
No worries!
It’s done when it’s done 🙂
Thanks for looking into it!
Thanks very much @tcrawley, merged and deployed & it’s working 🙌
My pleasure! I'm glad that worked.
There is a hidden rebuild button next to where it says scm info missing. I made it black in this screenshot but you should just barely be able to see it on hover.
Sorry! I found the button okay, but when I click it the page reports an exception
(No stack trace or anything is leaked, however)
(I know nothing just knew about that button) Did you look at the section of the FAQ about scm issues? https://github.com/cljdoc/cljdoc/blob/master/doc/userguide/faq.md#how-do-i-set-scm-info-for-my-project
I think I got the SCM working - but to test it I need a rebuild... But if I click rebuild, it seems to cause cljdoc to hit an exception and crash. Presumably it's being logged somewhere
You'd need to push a new version for it to pick up that information
For it to pick up new SCM info?
Maybe not a new version since it is a snapshot. But I really have no idea. Maybe someone more knowledgeable will have to reply.
Cool cool. Thanks for the help anyway 🙂