What is Reddit's opinion of
Julia?
From 3.5 billion Reddit comments
100 reviews of this app found across Reddit:
Homoiconicity just entails that programs are represented in a data structure of the language itself. In Lisp that's just lists — the data structure for everything. In Julia, there's an Expr type that represents Julia expressions. You can really easily generate Julia code in Julia code — we do it all the time and use the facility to do things like generate bindings for external libraries, reducing the repetitive code required to bind big libraries or to generate lots of similar routines in other situations.
You can read more here: http://julialang.org/manual/metaprogramming/.
For those who don't know Julia: It is a very performant JIT-compiled dynamic programming language targeted at scientific and technical computing (see http://julialang.org/ for more info).
Although the number of breaking changes in Julia's major releases is still huge, I am optimistic that in a few years it will become a good alternative to Matlab, Python and C in high performance numerical computing.
Julia is also a supported language of Jupyter Notebooks.
Python+NumPy+SciPy+matplotlib is an amazing combo for scientific computing. Whenever I have the freedom to choose, I use that over MATLAB. Python is a much nicer language to work in. Plus, NumPy's idioms are fairly easy to pick up if you already know MATLAB: http://www.scipy.org/NumPy_for_Matlab_Users
As for Octave, it works though as a programming environment it is pretty average. It is a good choice if you need to reuse an existing library from MATLAB. Unfortunately, Octave's implementation tend to be on the slow side—i.e., it is generally much slower than Python's or MATLAB's.
I have never used it, but I've heard good things about Julia: http://julialang.org/. The syntax of Julia is similar to MATLAB, but it has extra features to make general programming easier. Plus, they have a fast JIT implementation.
Anyhow, it is hard to make more specific recommendations without knowing your needs. If you are just looking to do straightforward numerical linear algebra, then above are all pretty good choices. For more complex tasks, like the ones we can find in computer vision, machine learning and material sciences, there are better alternatives.
One of the important feature of Julia is that it combines the easy syntax of python with the running speed of C. (If you didn't understand the above statement) : Python is easy to learn and program, but python itself is written in C, which means your python statements are processed in C and then to binary and run, hence python is an interpreter (reads and runs commands one by one). On the other hand, C is as close to machine as you can get. C being a compiler processes the entire code and translates to binary, but this speed comes with certain caveats. Since the code is translated in one go, you need to clearly specify which variables hold which type of data (hence you need to declare variables as int, float etc). This is not an issue in languages like python because it is essentially a C wrapper and hence stuff happens under the hood. I too don't know how Julia does it, so please refrain from asking me.
More details : http://julialang.org/
Julia seems like an interesting language, but one downside is that they claim to have macros that can solve the hygiene problem, but in fact they don't. They use gensym to generate unique identifiers, which is not enough to be truly hygienic (this is important if you want encapsulation and composability for your macros).
Another downside is that one of the only reasons I ever use R (some aspects of the design of R don't make sense; see this paper) is that it has a huge number of great libraries like ggplot2 or reshape.
My favorite language? Probably C++ / CUDA. I use it all the time for research and it's growing on me.
The language I want to like the most? Julia. It's an up-and-coming JIT compiled language that has the simplicity of python, but boasts of speeds closer to C++ (if you write well optimized code). I would definitely suggest taking a look at it, if you are interested!
And they are still relatively slow. The speed difference between one C program I have written and the equivalent one in Python with Numpy was significantly higher than expected. The Python program was more than 10 times slower. Using numpy for this particular problem wasn't a perfect glove, but using it was better than the regular Python performance.
This is actually why I'm really excited about Julia. The equivalent Julia program is less than 3 times slower than the C version but was very easy to write.
(Also, this is reddit so someone will probably mention PyPy: no, I did not try it for this problem)
yes, it is nice, but it is not attractive for newcomers. Just compare how a web-site for Julia language looks like or the Ruby or even Python.
I understand that visual appearance is kind of a dogs and whistles thing that has no influence on how great the language/system is, but it is very important in attracting newcomers and making the community around it viable and vibrant.
It's not an answer to your question, but I would recommend to take a look (if you haven't already) at the Julia Language. It is not a Lisp, but it's parser is implemented in Lisp (femtolisp) and it is homonoical, which makes metaprogramming almost as easy as in most other Lisps. It is certainly influenced by Lisps, but I am missing s-expressions a little bit (you can use them, but it is too verbose to be used in regular coded except in macros). But if you are from a python background, the syntax will look familiar to you. And even though it is intended to be a general-purpuse programming language, Julias's main purpose is high-performance numerical work. The community is very active and there are libraries for most numerical needs (including machine learning). You can easily execute fortran and C code from julia and python modules can be also called with the PyCall module. However, Julia is still very young and still has a many rough corners. New libraries are appearing all the time, but it is still little to what you have available in python for example.
Well, if you want a "real" Lisp for number-crunching I would use Common Lisp. I recently got into data analysis in particle physics and I was surprised when I learned that "The Higgs Boson Machine Learning Challenge" was won by a program written in Common Lisp and this challenge was 2014.
I work professionally in probability theory, and I program on an almost daily basis.
My personal language of choice is Processing. It is derived from Java, without some of the annoying cruft (so long System.out.println!). It is able to load any java library, so you can get all your fast linear algebra or FFTs as well as you can in any language. It is fairly speedy and cross platform. Perhaps most importantly for me: it has a very extensive set of easy-to-use graphics routines as it was actually built for data-vis people. Since I study various spatial stochastic processes, this makes it very easy for me to run nice big simulations and visualize the results in real time.
Another I have used extensively, and might be better if you are just starting out is Julia. It is a math specific language, cross platform, free, and often times can produce code which runs as quickly as C code. It has a very nice IDE. The one downside is that it is still a young language, so every so often you'll encounter something you want to do for which there is not yet a library.
Two points: > unless the problem you're simulating is fairly simple.
Solving a 1-D Burger's equation is fairly simple.
> Matlab is horribly inefficient
Written improperly, it is; With proper use of vectorization it is typically as fast or only several times slower than C. Take a look at the table with run-time comparison of languages on the Julia webpage^[1] . The problem is that most people get exposed to Matlab for a semester and make glorified calculators in it with for() loops galore. Properly vectorized, your expensive code is basically just optimized BLAS calls.
Short of production grade implementations I think Matlab is a very good choice as it is the appropriate domain specific language and permits very concise code: I can write a 1-D heat conduction solver in one line of Matlab. My entire Master's thesis on a high-order Discontinuous Galerkin solver for vortex methods works out to ~150 lines. It ran this sixth-order simulation with ~170k DOFs in 2 hours on one core of an Intel i7-3770.
_
^[1]Note: ^Their ^parse_int ^and ^rand_mat_stat ^implementations ^are ^slower ^than ^they ^should ^be ^because ^they ^didn't ^properly ^vectorize; ^I'll ^be ^submitting ^a ^pull ^request ^with ^vectorized ^versions ^that ^match ^the ^compiled ^languages' ^performance. ^The ^idiomatic ^recursive ^fibonacci ^implementation ^they ^used ^will ^be ^slow ^because ^their ^is ^a ^larger ^overhead ^for ^function ^calls, ^obviously ^a ^recursive ^approach ^is ^the ^wrong ^one ^there, ^but ^is ^a ^fair ^comparison ^nonetheless.
Seriously, this is all on the front page of the Julia site: http://julialang.org/
From the front page:
- Multiple dispatch: providing ability to define function behavior across many combinations of argument types
- Dynamic type system: types for documentation, optimization, and dispatch
- Good performance, approaching that of statically-compiled languages like C
- Built-in package manager
- Lisp-like macros and other metaprogramming facilities
- Call Python functions: use the PyCall package
- Call C functions directly: no wrappers or special APIs
- Powerful shell-like capabilities for managing other processes
- Designed for parallelism and distributed computation
- Coroutines: lightweight “green” threading
- User-defined types are as fast and compact as built-ins
- Automatic generation of efficient, specialized code for different argument types
- Elegant and extensible conversions and promotions for numeric and other types
- Efficient support for Unicode, including but not limited to UTF-8
- MIT licensed: free and open source
What's not to love in that? I've really loved Python over the years and it obviously has a rich ecosystem around it, but the basic inability to deal with performance issues has left a rather large opportunity available for a new up and coming language. This is it, it would seem.
Speaking of which, it would be nice to see some new posts in that Julia blog. Year 2012 has 7 posts, 2013 has 8, and 2014 has only 3. Someone might get the impression that the wonderful enthusiasm shown in "Why We Created Julia" has waned already! A suitably overenthusiastic post on staged functions or some other must-have feature of 0.4 wouldn't go amiss.
Julia (http://julialang.org) is MIT's open source attempt at the control/DSP and all linear algebra numerical computations. As a language it have many advantages over Matlab not tied by backward compatibility. OTOH, being a relatively new project, its ecosystem and library still not as comprehensive as Matlab. It's moving at a steady pace though.
C is the classic option for someone wanting to make fast code. It's fast, and definitely worth learning, but can be a lot less convenient than Python. The C language family includes some other languages as well, with the most popular being C++.
Java is widely used, and higher-level than C, and these days can be competitive in speed with C.
Julia is a nice up-and-coming language with quite respectable speed and a bunch of useful features, including the handy ability to import Python modules. I would recommend at least looking at it; it'll probably be the easiest of these to learn, and (once learned) the most convenient to use.
There are some other options, of course. Did you have any particular applications in mind?
Not an expert, but Erlang (with which Elixir shares a VM) is generally not used for parallel computing (in fact, Joe Armstrong, one of the creators of Erlang, specifically says in his book that Erlang isn't meant for fast numerical computations), rather it is used for applications where concurrency and responsiveness are advantageous. If you want something with good support for parallelism, more targeted at scientific and mathematical applications, you might look at Julia (http://julialang.org/).
Not to say you couldn't use Elixir, it just might not be the best choice for your use case (from how you describe it).
A language to keep your eye on is julia; it's targeted for the same audience as matlab, but designed elegantly learning from python, lisp, and quite a few other languages. It's also jitted with llvm, and can run code often times as fast as c. While it doesn't have nearly as many libraries as matlab, it has a very active community that are quickly working to fix that.
Off topic: The top comment there is written by Stefan Karpinski, one of the co-creators of Julia. Julia's a language focused on performant numerical programming, and I've been meaning to check it out for a while. It's got some useful language features and looks promising to me as a fast alternative (or complement) to Python.
At least for certain operations, it gets a lot of that for free, and so can NumPy by using MKL. Furthermore, sometimes NumPy is faster than MATLAB.
There are certainly cases where MATLAB will outperform NumPy, but there are a decent number of things NumPy or Python will do faster than MATLAB (admittedly that last one is promoting use of Julia which is a MATLAB-ish LLVM-based clone).
När jag först blev bekant med Julia tyckte jag mig se att vi hade en framtid tillsammans, men sedan krossades mitt hjärta då det visade sig att Julia använder 1-indexering. :-(
I have another option to suggest to you: Julia
Julia is a relatively new open-source (edit: meaning it's free) technical computing language. It has a REPL interface, and (i think) uses just in time compilation. It's easy to use, intuitive, and very fast. Their benchmarks say that it's almost as fast as Fortran or C, and my experience agrees with this. There are already quite a few libraries available for it, and everything that you need for basic math stuff (particularly linear algebra and matrices) is built-in.
The syntax is something like a hybrid between Python and Matlab; it's meant to make working with mathematics as easy as possible without having to sacrifice performance. I like the fact that it's easy to use it like a conventional programming language (like Python, and unlike Matlab), but it also makes the math really easy and intuitive (like Matlab, and unlike Python).
I can't do much to sell you on technical specs beyond what I've told you already, because I don't actually know a lot of the details about it, but that's why I like it so much: it's free, it's very fast, and it lets me think entirely about the math that I'm trying to do instead having to think about the programming. I can't recommend it highly enough.
Also, don't use Octave. It's way, way too slow.
Edit: oops, I forgot to answer the real question. No, I'd say the non-free programs aren't worth it, especially if you're already a proficient computer programmer.
I do most of my programming in python, but I'm looking around at the next-generation languages to see what I'll eventually use as its successor in my day-to-day programming. How does Ritchie stack up against a language like Julia? Any significant benefits I should know about?
I think that it is fine to be critical, but your style in doing so is different in a couple of respects from what I'd recommend.
One potential issue is that you have a tone that can be interpreted as challenging on a personal level. Perhaps you were trying to say that I've just not seen Julia's limitations because I've not looked in the right places. But overall, it sounded as if you were saying that I'm just not sophisticated enough (as a programmer) to see the problems and that my work is just sloppy one-off scripts. I chose to ignore that tone and carefully avoided acknowledging or engaging those points in my reply. In general it's counterproductive get into shouting matches over skill, competence or character.
Perhaps you should be more specific in your comments and try to engage with Julia on Julia's terms. Those comments of yours that I've seen here are very sweeping and sound as if they come from an outsider who hasn't, say, read any of Julia's design papers, and is trying to make Julia look like some other language (Python perhaps). Broad generalizations not sustained by an understanding of Julia's design goals are liable to fall on deaf ears. Please note, I'm not saying that you aren't knowledgeable of Julia and an experienced developer. It's just that you haven't make your level of Julia expertise and engagement clear to me in the comments that I've seen.
Try to think about how a complete stranger might view your comments. Try to make that stranger your friend.
You could use python+numpy/scipy(I think opencv bindings in python are available too), or use C/C++ & Opencv. There's one more newer language which is like mathematica, and has a similar interface available too, see http://julialang.org
Q: Tools
- Most interviewees mentioned how fast the tools were changing and how much learning the job required because of this.
- Several built their own tools because of very specific use cases that they need that commercial or open source tools didn't address.
- Programming languages - a few were looking into Julia (http://julialang.org/) at the time of the interview because of the speed and the expressivity.
Related question: has anyone checked out Julia?
It's an open-source language that's aiming to have the same functionality as Matlab (among other things).
I was wondering if anyone could see it replacing Matlab in the future (maybe within a decade)?
Looks like a pretty good start. I have a few basic questions and comments:
How do you intend to differentiate between prefix and infix notation? I presume
- 1returns −1 and- 1 3returns −2, but does- 1 - 3mean(-1) - (3)(−4) or1 - (-3)(+4)? Or is that a syntax error?What are your plans for the type system? I presume you’re going to differentiate objects into more types than just
obj..jlr(J u L ia R) would make more sense than.jrlfor the file extension.Assuming it’s meant to be pronounced /ˈdʒuːli.ə(r)/, it’s unfortunate that it has a nearly identical pronunciation to Julia, particularly in a non-rhotic accent.
I wonder what your Matlab license situation is, seeing that you are at a high school. A potential alternative would be the Julia language. Free, open source, performant, and similar syntax to Matlab. For a calculus course, there wouldn't be any features that would absolutely require Matlab itself. Of course, if you have plenty of licenses for your students, then this isn't a big deal.
$ julia _ _ _ ()_ | A fresh approach to technical computing () | () () | Documentation: http://docs.julialang.org _ _ _| | __ _ | Type "?help" for help. | | | | | | |/ ` | | | | || | | | (| | | Version 0.5.0 (2016-09-19 18:14 UTC) _/ |_'|||_'| | Official http://julialang.org/ release |_/ | x86_64-apple-darwin13.4.0
julia> @time using Plots, DataFrames 1.183326 seconds (1.38 M allocations: 63.092 MB, 3.66% gc time)
julia>
My computer's really old. Well, mid 2011 iMac. But 1.18 seconds isn't unbearable... :)
> I rewrote a VBA text parser that someone had in Julia and got a 10kX speedup
Impressive! I wasn't familiar with Julia but very much liked what I saw at their website. Thanks.
Since I'm a Julia developer, that's the language I used along with many of its packages. You're right, though, Python probably would have worked similarly. I just don't know its ecosystem as well.
Thanks, that looks interesting. Though I admit I'm pretty much sold on using Julia for data processing and numeric stuff. More because of the libraries than the language itself, of course, but I can't help but to notice an amazing amount of good design hidden underneath the dynamic typing and superficial similarity to Matlab. I was really caught off guard when I found (applicative) functors in there:
> s = slider(0:.1:1,label="Slider X:") > display(typeof(signal(s)); Reactive.Input{Float64} > xsquared = lift(x -> x*x, signal(s)) > color = lift((x, y, z) -> RGB(x, y, z), r, g, b)
where the last command can be abbreviated with a macro:
color = @lift RGB(r, g, b)
You can check out Escher if you are looking for a Web UI. https://github.com/shashi/Escher.jl
There are a couple of blog posts on the same by Tim Holy, although it is kinda old. http://julialang.org/blog/2013/05/graphical-user-interfaces-part1/
It surprises me that nobody has mentioned Julia: A fresh approach to technical computing.
Julia has a syntax that is similar to MATLAB, although not exactly a copy.
Learning Julia: http://julialang.org/learning/
I am the author of LLA — I found this thread because /u/vityok opened a ticket. LLA is currently semi-abandonware: I fix bugs, but don't plan to do anything more substantial. The reason is that I no longer use CL for this purpose, I switched to Julia.
Before writing LLA, I thought that the only obstacle is the lack of a library, but then I arrived at the conclusion that CL is not especially suitable for matrix manipulation/linear algebra. People try from time to time, since CL is such a great language and it would be so nice to use it for everything, but in the end almost all efforts are single-maintainer and eventually abandonned.
If you want to use matrices, vectors and linear algebra, but at the same time get some of the elegance of CL, I would recommend that you look at Julia. The language is still in flux, but has multiple dispatch, macros, and a lot of features inspired by/similar to CL.
It depends on what you want to do. Do you want to do general-purpose programming? Implement some mathematical concepts? etc?
Since you're coming from math, one of functional programming languages, as /u/lneutral mentioned, might be of interest. I would recommend Haskell or Erlang, since their notation is more math-like. Haskell is more pure, which makes it more math-like, but harder to program applications in. Erlang is more pragmatic, but has a concurrent programming focus, and AFAIK is slower for doing calculations.
Julia is another functional programming language, but with a more traditional syntax, and designed for technical computing. It is a fairly new language, so it may be incomplete, but it has many good features.
Python is a good general purpose programming language, is well-known and widely used, and is easy to learn. It also has some good libraries for scientific and numerical computation (numpy and scipy). Learning Python will also allow you to program with sage.
A lot of mathematical code is written in Fortran, so you it may be worth learning that if you want to read other people's code.
Julia's pretty damn high performance. It's got a JIT delay, sure, but once it gets going it's great.
Nim's entirely compiled, BTW.
edit: I feel I should point out that LuaJIT's performance is also really good. Lua can be pretty quick if you're careful with how you use its data types. I've gotta say it sometimes feels like using Crayola scissors sometimes, even though I like Lua.
From the first line of the Julia webpage: "Julia is a high-level, high-performance dynamic programming language for technical computing, with syntax that is familiar to users of other technical computing environments." In other words, the current focus is indeed on academics and data scientists.
Also, I don't think you really intended this so literally, but I wouldn't say that "Matlab users have for years and years" been "mostly focused on achieving better computational performance." Matlab is a high-level language for rapidly developing algorithms, some of which run quite fast. Matlab's performance is acceptable to many, but except for code that's mostly linear algebra, there are usually alternatives that can do better. In development time vs. execution time, Matlab favors the former.
Main appeal is higher performance. Both Python and R are quite slow. That's why some people are trying Torch, because LuaJIT. There is also Julia, a completely new language designed with higher performance in mind.
While LuaJIT and Julia are amazing, Rust is faster, and if we can match, say, Lua's level of interactivity and writability, Rust can be a better alternative.
In the best-case, really, really close. That one is kind of cheating though.
On a fairer note, Julia claims to have really good performance and uses bytecode; see some benchmarks. Julia is a JITed language specialized for scientific computing.
Perhaps Julia is worth looking at, then? I don't know about TCO, but it does offer the ability to specify types, and doesn't use GIL at the moment (last I checked). Syntax is a bit different from Python (it's not strictly indentation-based), but is close enough that it's a doable switch. It also has a good REPL, like Python does.
Personally, I don't mind dynamic typing in languages like Perl, where the explicit goal is to prioritize programmer-friendliness and expressive code over performance, but I've come to appreciate the ability to combine that expressiveness and the ability to statically-type things like Julia does rather nicely.
Sorry about that, perhaps 'glue' wasn't the exact right term in this case. Couldn't think of a better description though, using Julia to run routines on 3 different systems.
Here's a blog post from the main Julia blog that covers calling C from Julia.
> Do you know any languages off hand that efficiently execute multi-method dispatch over deeply parametric identityless types? :-)
I think I read some of those words when reading through the Julia docs :P.
I just always make sure I have appropriately named variables in my workspace as I work in the editor. It allows me to do quick spot checks of the code and enables tab completion.
But you may also be interested in julia.
The article is a bit sensationalist and doesn't do anything to convince me why I should care about this language. Looking at the webpage this seems like somewhere between matlab, Python and C. It's naive to think that one can create a programming language to suit all needs, and will inevitably result in an unprincipled mishmash of features as everyone and his dog asks for the language to support dependently-typed-aspect-oriented-as-a-cloud-service.
Haskell is probably the wrong choice for productivity and workflow integration in this area, while I highly recommend it as something that may open you to the frontiers of computer science, the conceptual hurdles may inconvenience your primary goals, as well as having unconventional notations for arrays and matrices.
Python+NumPy/SciPy has a wonderful ecosystem, and despite Python being slow on its own, is probably the best option for speed and convenience (combined with Numba or Cython), but has limited support for functional programming and isn't pretty under the hood or when code gets complex.
Have you heard about Julia? It's new, supports the entirety of lisp-style functional programming and has a syntax that takes the best from MATLAB and Python, and is reasonably fast and growing rapidly. If you know Python's matplotlib, you even use it within Julia.
For more: http://learnxinyminutes.com/docs/julia/
Too busy doing... other stuff. Projects, reading about electronics, programming a C (headers at least)parser and FFI-er for Julia in-Julia. Trying to design and make a 2.5D cart. Pining about getting a 3d printer, and all sorts of other things that i should nearly just as much not be doing as watching the full video. Nearly, well, i do prioritize... a bit.
The hype sounds fine, but I'm not sure if I like this part:
I noticed that you can put the value of a variable into a string by using '$' and the variable name (like a shell script)
"$greet, $whom.\n"
where 'greet' and 'whom' are variables (see http://julialang.org/manual/strings/).
So does that mean that I have to clean all input text to check for '$' and do some kind of data cleaning? I would rather have my text be text--I think the $ stuff introduced ambiguity.
I understand exactly what you're saying: I could never get into scipy/numpy because I dislike it just like you dislike Julia. I guess it's impossible to please everybody: to me Julia just clicks. It's close to my ideal numerical language.
And allow me to insist that it may be worth it to keep an eye out on developments: things like syntactic loop fusion (http://julialang.org/blog/2017/01/moredots) are (to me) extremely elegant, powerful and not available in many languages.
Depending on how fast you need it to be it might be worth giving Julia a go. You probably won't get Fortran/C++ speeds but it'll be much faster than python (even pypy) with a syntax similar to Matlab.
Python har også SciPy, NumPy og matplotlib som alle nok er interessante for OP. ipython notebooks virker som fancy greier.
Ellers fins det også et språk som heter julia som kan passe OP godt, men python har vel mer bread&butter-potensiale.
Julia does this quite nicely:
julia> bits(-25) "1111111111111111111111111111111111111111111111111111111111100111"
julia> bits(25) "0000000000000000000000000000000000000000000000000000000000011001"
julia> bits(Int8(-25)) "11100111"
for more: http://docs.julialang.org/en/release-0.3/stdlib/numbers/?highlight=bits#Base.bits
I've tried getting econ phd students interested in Python for scientific computing. They mostly acknowledge that it would be ideal to switch, but they're deterred by the startup costs of learning a new language.
I have my money on Julia as the heir to the scientific computing throne. It was created to replace the "prototype in matlab/python/r, then write in C/fortran" workflow -- it has the speed and the user-friendly syntax (also, loops are as fast as vectorization!). Just look at the very highly regarded Quantitative Economics site: it started out as a series of Python lessons, and now they have a Julia track.
If Python can't kill matlab, I think Julia ultimately will.
Let me introduce you to Julia. If you can get over 1-based indexing, Julia is essentially Matlab minus all the problems, with a strong type system, magical macros, and if something hasn't been implemented yet you can typically import a python package and treat it like native code (thanks to the magical macro system). Take a look: http://julialang.org/
An alternative is to use a language that supports arbitrary precison computations, such as Julia (http://julialang.org, also see the section on BigFloats in http://docs.julialang.org/en/release-0.3/stdlib/numbers/ )
Julia!
It's a lot like Lua, but much faster, a huge standard library, and some very cool and special features, such as defining functions like this:
>f(x) = x^2
and also writing julia functions directly in C, or Assembly, right in your normal code.
Scientific programmers are using more than one language/framework. A language that could become important is Julia. But due to a lack of libraries or statistical utilities it's recommended to also learn at least Python and R. Python is a good start.
Python, Matlab and R all have excellent tools for machine learning. I personally prefer Python because it's reasonably well designed as a language in its own right. The others seem like they're a bit cobbled together to expose the toolboxes and libraries they have. Python also has scikit-learn which is wonderfully well documented. I guess it depends on what you have available (I know Matlab costs an arm and a leg whilst the others are free) and what you're most comfortable with.
There is also Julia which is the new kid on the block. One can use Python libraries from with in it so that might be worth having a look at too.
Or http://learnxinyminutes.com/docs/julia/ I've heard a couple people say that Julia has a very pythonic feel to it; At least within numerical computing it writes a lot like pseudo code, is easy to read, and is as flexible as Python is with monkey patching. But it does it all without monkey patching, or most other impossible-to-optimize features, and as a result is many fold more efficient than Lisp or PyPy. This paper in particular was absolutely fascinating to me on how to keep a language' semantics efficient to execute, while having better 'ergonomics' than C / Fortan: http://julialang.org/images/julia-dynamic-2012-tr.pdf
I got going doing some data analysis using JavaScript (node). I prototyped it relatively quickly. Then I rewrote the inner analysis functions in C++, it ran a lot faster. Then I used some SIMD intrinsics and it ran much faster still.
Optimized C++ code, in my experience, runs massively faster than in node. I'm calling it C++ but it's C code which uses node bindings. The C code is very similar to JavaScript that's written to run quickly, the main difference is in having types. I found it easy to translate between JavaScript and C. I don't think that a programmer who can understand optimized JavaScript would struggle to read the same thing in C. This reasoning applies to what the author said about Python and numpy too. If you want fast computation from JavaScript, use some linked compiled code. At least I have not ever been able to run JavaScript that's in the same ballpark regarding speed.
It probably depends on how much computation must be done. If it's 'only' requiring a billion calculations you should be fine with JavaScript. If higher speed translates to a better analysis, I think that other languages are worth looking into. I like using node/iojs, and would prefer to be writing JavaScript that runs very fast indeed, it's just that I can't do that, so I use C/C++ instead.
While I found porting JavaScript algorithms to C to be easy, learning how to code the node/iojs addon system was not. In the end I used NAN to help binding the C++ with the JavaScript.
Julia may be a better choice for those who want to do data analysis in a high level language. I've not used it, but it's got nice syntax and according to benchmarks is faster than V8 JavaScript. It's still slower than C (in most cases).
I'll be using JavaScript for some parts of the data analysis stack.
V8 is a pretty damn impressive JIT (thank Google Chrome for that), and libuv is a very nice abstraction layer for cross-platform asynchronous IO. Node takes advantage of those to provide that combination of ease-of-use and performance, but there are other recent languages that also provide the same combination, depending on your application domain - Go, Julia, maybe Rust - see some microbenchmarks [admittedly from a biased source] here
Si t'aimes bien ça il y a le langage Julia aussi qui est en plein développement. Ca sert entre autre à faire des graphes interactifs en ligne mais il y a plein d'autres avantages.
As I understand it, much of the interest in Matlab is for the toolboxes, while there is less love for the actual language.
There is remarkable buzz surrounding Julia, because people have achieved performance benchmarks similar to C and Fortran while the language itself is more amenable to scientific and technical computing by making abstraction easier.
There are a lot gushing blog posts around the web explaining why there is so much excitement about a language that has only been around a couple years and is still only at version 0.3.
You ought to be familiar with the project if for no other reason than the benchmarking they did. You may be familiar with John Myles White, a facebook data scientist that is one of the main contributors and proselytizers of the language, who wrote Machine Learning for Hackers.
-- Also a Data Scientist
To be honest OP- Unless there are some serious funds for top notch developers, programmers, QA, etc. then this does not seem like the best idea.
Developing a full ERP system is no joke.
That said if you want to do some simple statistical gymnastics with finances I would check out Wolfram Alpha and Mathematica for more of a fully functional language.
If you want something that is a cutting edge language that can take advantage of distributed systems- Julia may be the way to go. But it is still in development.
I'm curious what you want to do with it.
(For what it's worth, Julia seems pretty straightforward, and has good built-in support for multithreading and running on a cluster. Might be worth a look.)
I really like hmatrix, it makes matrix and vector operations simple but uses gsl/lapack/blas in the background so it is fast.
I don't quite see the appeal of data frames in R, the main feature I feel is missing in Haskell libraries is to operate on subsets of rows/columns of arrays using a simple syntax.
In R being able to do things like a[which(rowMeans(a)>0),] to select subsets of an array both as a destination to write to and to read from can make certain things much quicker to write.
Random-fu is incredibly useful for sampling from random distributions and nicely encapsulates the results as RVar's so that it is clear whether something is pure or a random variable.
An interesting alternative to R is Julia - http://julialang.org. It seems to fix some of the things that can be a problem in R. I'm not quite sure how stable it is yet though.
My research is into an estimator model for a dual solenoid - using the voltage and current measurements to estimate the position of the actuator. Removes the need for a costly LVDT.
As a side project, I'm currently working on a control systems library for Julia. Just started, and would love some people to work with. Never really done open source before.
I haven't heard of Julia. Let me have a look.
> [Julia's performance] often comes within a factor of two of fully optimized C code (Wikipedia
Ah. So let's see:
- It's at best roughly 50% as fast as C code.
- It's only 2 years old, which makes it very immature. (For instance, a paper from the authors says, "Julia is not yet able to cache generated native codee, and so incurs a startup time of about two seconds to compile basic library functions.")
- It does support calling C functions, but the syntax for it looks painful.
- We'd have to port the compiler to the CPU architectures and OSes we support, which includes Blackfin and ARM CPUs, and Fusion and uC/OS-I, -II, and -III. And then we'd have to figure out a way to call Julia code from Objective-C and Java in order to support our iOS and Android ports.
Thanks, but no. We'll stick with C.
Unfortunately, I can't seem to locate the benchmarks I'm remembering; the julia-lang page is the closest I can find. Performance is abysmal for anything that's not matrix based, as you would expect, and performance versus matlab was 3-100x slower, IIRC.
If you need Matlab compatibility (as in your case), Octave is great. Otherwise, you're probably better served by python/numpy: performance is similar and there are better libraries, especially when you need to interface with the "outside," like a database.
As a side note, check out this comparison of the numerical performance of javascript (V8) versus R and Python. Suffice it to say that I was more than a little surprised. I guess Google's been killing it with V8 optimizations. The only slow number in that list can likely be attributed to not using highly optimized fortran BLAS/LAPACK libraries.
I heard never heard of Julia before. The benchmarks for it seem pretty impressive ~~and it is nice to see another language adopt significant white space~~.
EDIT: I was incorrect about the significant white space. At first glance it looked like it was using it, but I had just missed the end keyword at the end of blocks.
>yeah, I kinda blew it with the memory management. I'm not really that hardcore of a coder, but from what I understood C++ is a bit more flexible when it comes to memory management (because pointers), though I would need to be an expert in both the programming languages to be able to express a valid opinion.
Pointers are both a blessing and a curse. pointer aliasing is a problem for compilers when optimizing, which leads to certain use cases where fortran is faster.
>Numpy as far as I know is built on top of LAPACK, which unfortunately is a Fortran library (and yes, I do use Numpy instead of Matlab every now and then). Still, it's more than usable from C. Also, the bulk of linear algebra is usually handled by lapack (I'll concede that the only linear algebra I coded in C was the homework for several Numerical Methods courses).
Just about everything that does linear algebra is built on top of lapack these days, Unfortunately, there is a large fraction of the lapack library that doesn't have wrappers for C, which is kind of annoying... on the other hand, writing your own isn't that hard.
Weirdly enough, I actually use C++ for most of my programming needs (and would love for something like julia or D become more used in the physics/HPC world, unfortunately, the library support isn't REALLY there). I like C++ better, and I find fortran funky to deal with. But there are a lot of nice language features in the new versions, and if there were better libraries, I would be more inclined to use it.
I don't believe in benchmarks, but look here: http://julialang.org/ I would call it a kind of fast if it is not 5 times slower as c. If you try, as they did in the julialang thread, js can be pushed to be much faster with arrays (the last two benchmarks). DOM is slow, though.
From the Mathworks website
> For how long can I use student software? > >You can use student software indefinitely. After you graduate you may need to purchase a professional license.
You just won't get the annual update refresh.
To be honest, after having gone through this, if its just basic stuff, you'll be better off learning Julia as its mostly Matlab compatible syntax, and free. Python (and scipy / bumpy) is also a good option, but it takes much more of a learning curve coming from MATLAB style syntax.
Look very nice but I wasn't able to compile it on a fresh 0.6 julia (Ubuntu 16.04). Any ideas why?
lali@TITAN-L:~$ julia --color=no _ _ _ ()_ | A fresh approach to technical computing () | () () | Documentation: https://docs.julialang.org _ _ _| | __ _ | Type "?help" for help. | | | | | | |/ ` | | | | || | | | (| | | Version 0.6.0 (2017-06-19 13:05 UTC) _/ |_'|||_'| | Official http://julialang.org/ release |_/ | x86_64-pc-linux-gnu
julia> using GtkIDE
INFO: Precompiling module GtkIDE.
Reading package lists... Done
Building dependency tree
Reading state information... Done
libgtksourceview-3.0-1 is already the newest version (3.18.2-1).
libgtksourceview-3.0-1 set to manually installed.
0 upgraded, 0 newly installed, 0 to remove and 4 not upgraded.
(julia:7883): GtkSourceView-ERROR **: Error while loading the completion UI: .:23:106 Invalid object type 'GtkSourceCompletionInfo' ERROR: LoadError: Failed to precompile GtkSourceWidget to /home/lali/.julia/lib/v0.6/GtkSourceWidget.ji. Stacktrace: [1] compilecache(::String) at ./loading.jl:703 [2] _require(::Symbol) at ./loading.jl:456 [3] require(::Symbol) at ./loading.jl:398 [4] include_from_node1(::String) at ./loading.jl:569 [5] include(::String) at ./sysimg.jl:14 [6] anonymous at ./<missing>:2 while loading /home/lali/.julia/v0.6/GtkIDE/src/GtkIDE.jl, in expression starting on line 9 ERROR: Failed to precompile GtkIDE to /home/lali/.julia/lib/v0.6/GtkIDE.ji. Stacktrace: [1] compilecache(::String) at ./loading.jl:703 [2] _require(::Symbol) at ./loading.jl:490 [3] require(::Symbol) at ./loading.jl:398
julia>
Julia is a high-level, high-performance dynamic programming language for technical computing. It has libraries for DSP, stats, linear algebra, etc.
Can you open up a REPL and reproduce the brackets? I am surprised you're getting brackets in this situation. Check it out:
_ _ _ ()_ | A fresh approach to technical computing () | () () | Documentation: http://docs.julialang.org _ _ _| | __ _ | Type "?help" for help. | | | | | | |/ ` | | | | || | | | (| | | Version 0.5.0 (2016-09-19 18:14 UTC) _/ |_'|||_'| | Official http://julialang.org/ release |_/ | x86_64-w64-mingw32
julia> dict = Dict{Int, Vector{Int}}() Dict{Int64,Array{Int64,1}} with 0 entries
julia> for i in 1:3 dict[i] = [i,i,i] end
julia> typeof(dict) Dict{Int64,Array{Int64,1}}
julia> dict Dict{Int64,Array{Int64,1}} with 3 entries: 2 => [2,2,2] 3 => [3,3,3] 1 => [1,1,1]
julia> for el in dict[1] println(el) end 1 1 1
julia>
Edit: if you want to print the array on one line sans brackets you can do something like
julia> for el in dict[1] print(string(el) * " ") end 1 1 1
From the ones you mentioned Python+scipy would be my choice. Unless you plan on doing webdevelopment things there is little point in learning Javascript.
But at my university we tend to use Matlab/Octave, it's a scientific programming language that has a lot of stuff built in (like graphs and linear algebra). It may not have the best performance around but if you want performance you're probably better of writing C/C++ code (or using libraries written in C/C++). Though personally Julia is my favourite scientific programming language, cause it's quick, has built-in linear algebra (vector/matrix calculations) support, a built-in package manager and can easily use python/C/C++ libraries (it's still a bit young though).
I'm really hoping Julia takes off. I use Python regularly, but I still go back to MATLAB frequently because it's made with numerical computing in mind. If I ever have to do a quick analysis or something that isn't very computationally expensive, it's just so much easier to crank out something in MATLAB than it is in Python. It would be nice to see a programming language focused on numerical computing that isn't proprietary (and also is blazing fast).
Wasn't that sort of what Julia is meant to do? Or am I misinformed? I don't use either professionally, so I only know the high-level goals of both, but if I were tasked with updating code written in Fortran, Julia would be the first language I prototyped in.
You also have this kind of problem if you're implementing multiple dispatch or overloadable binary operators into a dynamically typed programming language.
In lever I implemented multiple dispatch & operators using a lookup table. It resembles bit of nasty problems if you're going to reload code. I didn't even try to solve it in presence of inheritance. That'd have been really inefficient in a dynamically typed language. The benefit was I made many operators into multi-dispatch -functions that can be directly referenced if needed.
And, if you say "interesting, that resembles me about Julia that is using multidispatch everywhere, do you know it?" Yes, I've read about Julia. I considered doing everything a multimethod, but I do feel there are very many functions that only have one sensible context where they work.
Python has lot of libraries like pandas, numpy and matplotlib for analytics. Pandas and numpy are written in c. so speed should not be an issue. Golang ecosystem may take few more years to catch up python. Continuum has anacoda with all python packages bundled for anaytics. https://www.continuum.io/downloads. You can try julia language too for analytics. It is still young but promising. http://julialang.org/ .
As someone who has never used it, I'm curious to hear what you think about Julia Lang? It was the first thing that I thought of when I saw your tag line. Is it too directed at scientific computing and you are trying to do something which is more general purpose?
To complete the list of languages used in Data Science / Scientific Programming (areas that usually deal with real-world numbers alot), besides Python, Matlab, R there would also be the newcomer Julia. Sometimes also Mathematica, and Wolfram Language (very expensive). For high-performance requirements Fortran and C. Sometimes also C++ is used, especially when the features of the Root library are needed.
They are all more or less equally hard to learn. The first ones are a little bit simpler. You need to learn logic - which is equal across the whole programming world. Syntax is just a matter of days, if not shorter. Standard libraries are something to take into consideration, they may differ heavily.
A physicist friend started with Matlab, and could get done work quickly. Though limitations of the rather bad implementation were also hindering him very quickly. After some calculations he needed to repeatedly plot and export to PNG, which needed like 30 minutes each time for his data sets. Exactly the same task in C++ with Root, <30 seconds.
> MATLAB
I hope the only way Matlab defines the future of coding is by being put in some sort of a museum of horrible mistakes. If people really insist on doing math in dynamically typed languages, why not Julia?
The language looks great! I also like the expressiveness of it very much :D But i have to say, many people don't like having either to explicitly define returns or have it implicit from last expression ( what happened to coffeescript ).
I'm also very passionate about functional and expressions in languages, a big fan of Julia which follows that nicely, but even following functional concepts you don't really need the return to be explicit, you can just have an undefined returned, as majority of industry/commercial programs are about taking steps to process data where some steps are at the end of processes and don't need to return a value.
R and Julia are two options that are specifically designed to fill the same niche as Matlab - if you're just looking for alternatives then maybe start there.
If you're looking for a language to complement, rather than replace Matlab then pick any of the mainstream languages from the FAQ, depending on what your goals are. Matlab is good at data processing, but perhaps something more front-end oriented like JavaScript or C#?
What is the relation to the Julia language project, from which you've lifted the sans serif/colored dots logo?
In the data science branch you should not limit yourself to one language.
In addition to Python and R, Matlab won't hurt. Also, a nice new language in that area is Julia.
Also, have a look at those resources: http://datasciencemasters.org/ although, you'll probably see most of it anyway in your courses.
People are using these techniques to analyze for example the stock market, consumer behavior, for general market research, for advertisement etc.
Sorry, I don't really know what you mean with "humanities-oriented IT-masters degree where design, governance, innovation and stuff [...] takes up the majority of the time".
There are actually several interpreted languages on my list: Scala, C#, F#, Julia. It's true that only Lua and Julia are dynamically typed, but dynamic types are cheap when you have a JIT.
Lua's actually more like the C of dynamic, interpreted languages. It's not particularly expressive and it's too minimal to be truly convenient, but it's small, really fast (with the right interpreter) and embeddable.
Lua is a lovely target for a JIT because its semantics are so simple - I've heard that LuaJIT can finish JITing a loop in only 50 iterations (as opposed to, say, 1000+ for a typical Javascript interpreter). This makes it really nice to use where the latency of JITs are problematic but you still want runtime dynamicism - game scripting engines are a good example.
Julia's perhaps a better example, though. It was designed from the ground up to be fast, and it is. See the homepage for examples. This is because there are certain abstractions that JITs are really, really good at optimizing. By focussing on these parts and avoiding the harder bits, you can build a system that's completely dynamic but blazingly quick at runtime.
Well in the newest system it doesnt have a package system, and i like it that way! That said, wish it had macros so some simple shit could be done to make it slightly less repetative. I.e. @localize_members table (membername, membername2) to "import".(though it can also used for tables anywhere!) (in Julia syntax, probably they should just take that if they implement it)
Really, i love the whole "everything is tables" thing. To be clear "importing would replace:
local package = require "some_package" local a,b = package.a, package.b
With @localize_members require("some_package") (a, b) (edit: this is akin to common lisps with-slots)
> Some of its syntax choices are dubious, like providing break but not continue for loops. And yet providing goto.
Coming from lisp.. Well, dat not syntax. But yeah, perhaps a continue would be nice. Really, would like to have macros. Although inevitably that'd lead to people making silly ones.
> No baked-in OOP. OOP
Well, if it can be implemented in Lua, Lua should not bake-it-in. But then.. well, it would be nice to have a standard library or something. Seems like a risk of haphazzardness.
Building-it-in is like "blessing" it, and then force-feeding it to everyone, so that aint better.
> Batteries not included.
They clearly state that they're not into making a popular language, but one they'd like to use. Really though, why saddle all sorts of stuff with the distribution.
Why implement package systems when distros should be doing that already. Perhaps they should do it anyway though, because in practice, there are too many package systems and many of them wont have it.. Dunno.
One disadvantage is that Luas approach is very adverse to being compiled. At least, when it is not JIT, and when the user is not limiting him(her)self to a subset.
Python's much faster with numerical calculations than most people give it credit for. It could have been done in even less.
EDIT: weird that this became a controversial post for me. As an example, take these benchmarks from the Julia programming language website (and I'm not cherry-picking this one suite of benchmarks): http://julialang.org/benchmarks/
All of those benchmarks are measured against C (that is, how much faster or slower is the computation in C). Notice how Python, despite being given crap all the time for being "slow", but in practice certain operations tend to me much more costly than others and tends to be faster than languages that don't get the same sort of flak.
Check out this discussion on benchmarking NumPy specifically (NumPy, Pandas and the like is why anyone uses Python for computational things anyway, but the post we're referring to didn't specifically mention implementing the code in NumPy): http://stackoverflow.com/questions/7596612/benchmarking-python-vs-c-using-blas-and-numpy
I guess you're on Windows? There's been a fair bit of work on that. You could try is checking out a nightly build to see how much things have improved – if there are any more issues you still have time to complain ;)
To build on what csp256 said, I guess the sort of competition (games) that Julia programmers are interested in are those such as the following
- Competition between languages for the best speed-vs ease-of-use tradeoff
- Competition to make the best tool for optimization
- Competition to simulate quantum systems the quickest
You should check out Julia. Matrix manipulation built-in, beautiful clean syntax, functional language, REPL console, super-fast (approaching that of statically compiled C) and no need to worry about "vectorizing" operations because for-loops are just as fast...