core-matrix

intended for specific discussion around core.matrix (usage and development) For general data work check out #data-science
2015-08-26T03:40:26.000006Z

I think Neanderthal should simply become a core.matrix implementation. There's nothing stopping that from working as far as I can see

2015-08-26T03:41:31.000007Z

Spark RDDs could also possibly be turned into core.matrix implementations, haven't tried it yet but it should work

2015-08-26T03:48:01.000008Z

@tnoda Rashinban looks great! I think it could also work well with core.matrix, at least in terms of interop with matrices / vectors etc. The main thing to do is ensure that arguments to functions get interpreted using the core.matrix API, i.e. you use functions like clojure.core.matrix/to-double-array to ensure that any core.matrix array can be used as an argument

2015-08-26T03:51:30.000009Z

See also this issue, which I think would be helpful (for performance at least) - https://github.com/mikera/core.matrix/issues/247

2015-08-26T08:49:45.000011Z

hi, tnoda. thank you for taking the effort to bring Rs abundance to Clojure. Personally, I would potentially use it for comparing my own data analysis/machine learning algorithms written in clojure with existing R options. So, not for the main stuff, but still VERY useful.

2015-08-26T08:51:19.000012Z

Having that in mind, I do not care whether you base it on core.matrix or not, although I have to admit it would be best for MY use cases if you support Neanderthal.

2015-08-26T08:52:56.000013Z

Neanderthal could be useful for you because its main implementation is based on direct byte buffers, which would enable you to transfer data from Clojure to R and vice versa without any copying.

2015-08-26T08:55:05.000014Z

Yeah, mike would also say that he does not see why core.matrix cannot adopt the same approach. Well, in that detail he is right. core.matrix is (probably) capable of having a direct byte buffer based implementation, but somehow, aftew a few years of exisence, it still does not, so you'd have to write it yourself.

2015-08-26T08:56:16.000015Z

There are also some things in core.matrix design that made it unsuitable for Neanderthal to be based on, but that is another discussion that we already had.

2015-08-26T09:04:20.000016Z

BTW, in Neanderthal, if you do not use the native BLAS operations, but only use the direct byte buffer as a data structure, you do not even have to have ATLAS installed on your machine (which is the only one pain point for inexperienced users currently). Everything will work out of the box.