Clojurians Log v2

@micha im also thinking about these trust questions for many years. i've even participated in the http://cacert.org movement and got an assurer status > You have made 28 assurances which ranks you as the #1791 top assurer. > Total Points Issued: 901

i think the main problem is that trust is not a binary quantity and im not sure what the measure of trust should be. i understand proof of work is one proxy for this, but it's rather complicated infrastructure to setup, maintain, etc.

i have a gut feeling that mirroring how we gauge trust in real-world context every day would be a good enough solution. that could be done by 1. visualizing a trust network 2. calculate a trust score 3. allow hinting the calculation based on our preferences (eg. i dont really trust microsoft or verisign, but i trust micha) 4. visualizing/monitoring changes in the web of trust over time

it all hinges on what do we consider the electronic representation of identity of course

this was quite well summarized in this famous keynote https://www.youtube.com/watch?v=RrpajcAgR1E

so to me it's clear that identity can be a set of interconnected identity representations, which all add to the overall trust level

pgp signatures can prove that an avatar and an email address are indeed represent a real individual

imean i can just choose to accept it as a proof and for example i would sign your gpg key based on the fact i've seen you in youtube talks and we had skype video chats and conversations about specific commits which bear your github username.

to me having such proof shown from a couple of community members who also trust each other would make your software a lot more trustable than being signed by come centralized CA with questionable security practices and opaque internal processes...

but i think the main problem is the lack of good UX for all this in the 1st place as @danielcompton points it out on https://github.com/clojars/clojars-web/issues/560

iirc the debian ecosystem was quite successfully maintaining a gpg ecosystem for their packager

the 2nd problem could be the lack of protocol for representing such claims that i just made about micha for example. but let me know if you know about any good ones.

Agree on this latter point, and specifically, PGP done right works with actual I'd checks and of course is only an identity check. Then there is the "how much I trust you as a dev" part, and this can be a custom metric (# of project downloads on Clojars maybe) . This point for me is the part noone has yet solved, but GitHub is pretty much the implicit CA here. Everything coupled with ipfs to avoid dependency tampering (distributed, signed, high available deps)

im still not sure though how to trust the public keys from the various keyservers though... 😕

Keys could reside on ipfs as well?

Do you mean in terms of trust or tampering?

just trust

Yeah trust as a "good dev" is a difficult concept to measure, but there can be ways imho, we have so much info on GitHub/BitBucket..of course nothing really as accurate as coding shoulder to shoulder..

like if none of my trusted identities are transitively trusting an identity, how do i know it's not someone impersonating a specific email and natural-name combination?

well, i think trust is difficult to measure on a unified scale, BUT that's why in point 2 & 3 i proposed having parametric algorithms which everyone / every organization can customize/tune for themselves

they can combine extra rules, like github metrics into their /user or /org trust score formula and thats it

it helps to compare libraries against each other and also monitor over time the trust level of the various software components within the organization

Yeah well, impersonation is very very complicated to ascertain in the digital world..and maybe a bit off here, at the end of the day we want to first of all be sure that the coder is trustful and she is not including strange stuff in the oss library I am using..

Yeah I like the above

so we don't have to tackle the problem of globally understood and accepted trust score calculation...

yet it would be very useful and already better that what we have right now, which roughly equals to: 1. blind trust 2. trust central corporations because we have to pay them for witnessing information...

Agree, also successful peer reviews could increase the "trust score"

and what we havent talked about yet is digitally signed timestamps which should ideally accompany all signatures, so we can't cheat on time either...

it should be easy to setup a time stamping service just for being able to fully dissolve centralization, but a few years ago when i checked i havent found any easy solution to this either...

Uhm yeah nothing comes to mind actually, the signed timestamp is implicit when you deploy an artifact right? It would be implicit in ipfs

@richiardiandrea which part of IPFS ensures trustable time stamping?

(i still don't know the whole IPFS in and out, that's why im asking)

just to be clear im talking about https://en.wikipedia.org/wiki/Trusted_timestamping#Trusted_.28digital.29_timestamping specifically

so i imagine an interface which shows all of the players in a trust network, assigns default scores to them, but presents a list where you can override these. for example if something is timestamped with a privately operated timestamp server, i could say i don't trust it, but if i happen to trust the operator of it, then it's default wouldn't be "dont trust"

Reading

Oh ok that's a different UX, I was just thinking of an implicit stamped timestamp as part of the (immutable) metadata sent over with an artifact..I see you are talking about something else ;)

Regarding the security discussion: here's a specification for securing package indexes like Clojars. https://theupdateframework.github.io

And here's a summary about it in the context of Python: https://lwn.net/Articles/629426/

Other languages have investigated it as well - at least Ruby, Haskell and Rust - but I can't tell if any of the implementations is complete

I'm not sure how easy it would be to adapt to Maven repos, but it's worth taking a look.

A few thoughts RE: Security: 1. The repo is your only real source of truth, packages and even compilers can be hacked. 2. That means that the only way to really validate code is to a. trust a developer (and the access control on the repo) and b. content address the code to make sure your getting what the trusted dev was intending. 3. Ideally at the repo level trust is established socially via real world interactions which could also be related to online activity like github data (this is kinda what we already do, for better or worse) 4. In order to validate the packages content, you cannot receive the verification data and the package from the same source, this is a basic single point of failure in any architecture. It’s not that I don’t trust clojars, it’s that it makes them a single point of failure and also means they are the ones telling me what my trusted developers packages are. 5. Whats needed is a way to verify from the developer what the data should be (or from the repo directly) or any other distributed trust, and then verify that against the packages, in transit security is “good enough” already if you are using certificates 6. none of this takes into account the versioning or human readable layer to all this, versions would basically be sugar on top of all the verification, since it’s still needed in a package manager.

> Whats needed is a way to verify from the developer what the data should be (or from the repo directly) or any other distributed trust, and then verify that against the packages, in transit security is “good enough” already if you are using certificates Okay, as I have pointed out elsewhere, IPFS literally dogfoods itself in its source code to get upstream dependencies and avoids trusting certs - https://github.com/ipfs/go-ipfs/blob/0e2b4eb4eb68fab5497d0e8f8472d025b5c1d431/core/corenet/net.go#L7

I agree IPFS is probably the way to go

Now, registries can also be hacked, but you could write an append-only registry on a blockchain.

couldnt you just use ipfs as the registry

Yeah, they have a nameservice thingy

I'm not sure how to make it append-only

i think that would be more for the ipfs app to handle

IPFS isn't a consensus layer, it's a content addressable storage and delivery mechanism with a distributed hash table.

@xcthulhu there is this also https://github.com/ipfs/notes/issues/148

Yeah, it's a TODO

But as far as I know it's not part of the white paper or anything

Yeah seems very concept

You can already make a registry using a blockchain

It's just a lot of work to get integration.

wow, lots of activity around securing clojars while I was on vacation. That's great that folks are interested!

there's lots to reply to here, but a few initial thoughts:

clojars is currently a standard maven repo (from a read point of view). Moving away from that to some custom format or distribution protocol would mean providing plugins for any build tool that would access it (currently maven, boot, lein, gradle, plus scripts that wget/curl deps directly)

IIUC, IPFS would provide immutability and distribution, but couldn't guarantee that version X+1 was published by the same party as version X. All we can know is the version X that we pulled yesterday is the same as the version X we pulled today (which can only be done currently by keeping the hash of X local, so when you pull it again you can verify you are getting the same thing)

please correct me if I'm wrong, my understanding of IPFS is surface

we're working towards putting the repo behind a CDN, and have been working towards that for 9 months. That's a simple change when compared to moving to IPFS, so I can only imagine that transition would take years to implement

we actually don't want true immutability, given that there are legitimate reasons to remove artifacts (which are all handled by the admins, never by users directly)

we use gpg because maven does, but gpg is nothing without a WoT and a way to verify that (which has been touched on several times)

I think the best approach here is to enumerate the real problems here as gh issues (https://github.com/clojars/clojars-web/issues), without discussing technologies initially - we need a clear statement of what the issues actually are

and, tbh, if users have to understand IPFS and/or blockchains to build, then we're here: http://howfuckedismydatabase.com/nosql/fault-tolerance.png

@tcrawley are there really cases where you must delete an artifact after it's been published? like i see reasons why it's desired, but i'm not aware of any real need?

like if i accidentally push an artifact, like boot with version 9999.0.0 by accident

it's not awesome, but it's also just a number

i could push the next artifact with version 10000.0.0

@micha: the "must" case is when a user accidentally pushes private code or credentials in a jar

those generally don't even show up as gh issues, they mail <mailto:contact@clojars.org|contact@clojars.org>

I don't delete in the case of "version X is broken and I want to replace it"

ah, private code is an interesting case

credentials i don't really think is a problem that needs deletion because you must assume they've been compromised anyway. if you don't you're negligent imo

but private code though, that's interesting, or i guess code you aren't authorized to upload to clojars in the first place

yeah, good point re: credentials. I don't think we've ever had that case, it's always been code you don't have rights to release

i wonder if a cooling-off period would be sufficient?

like a level 1 cache that isn't in the immutable store for some amount of time

only visible to the user who uploaded the artifact or something?

perhaps like the "promote" button we currently have

i guess the legal situation maybe makes immutability a no-go fundamentally

(There has not been promote button in nearly a year)

oh wow

so it wasn't immutable

what wasn't immutable?

sorry bad joke

ah :)

the promote button was part of the trusted repo feature that was never finished and never used

it just led to confusion

that's how http://oss.sonatype.org works - you deploy to a staging repo, then manually release once you are happy with it

but maven users hated that step so much that sonatype released a plugin that auto-releases for you

clojars can still be directed by a court to remove something or sued if you can't do it

doing away with some of the value of the staging repo

so it seems that immutablility is a pipe dream for a real repo

yes, and at this point, that would mean I would get sued directly. which is fun to think about

THANKS FOR BRINGING IT UP

oh man sorry

:)

i'll send you a file in a cake

a jar file?

hahahaha

most disappointing mail call ever

jars got me in here, jars will get me out

heh

things like poms can be stored immutably though

or metadata

it's only the actual artifacts that are a liability, maybe

what does that buy us?

the ability to verify that an artifact at some URL has the right contents

and also the history of contents that have ever been at that URL

so you'd at least be able to know if an artifact has been modified

like if there was a repo-wide metadata file in git, sort of

when something is changed or added you make a commit

tools could then refuse to use artifacts that aren't in the metadata file or have the wrong hash or whatever

you could still delete the artifact from storage if you need to

without deleting the metadata from git, you just make a revert instead, so you preserve the ability to audit

@danielcompton nice summary

thanks