Just so. And there is often handwringing about packages like SPSS enabling the heathens to analyze data. I’ve always felt that such concerns were misplaced, because the heathens would otherwise be making decisions without any acquaintance with the data.

you are unlikely to be doing HLMs anyway unless you have direct access to God

I know the deity.interface group has stuff in progress, but it isn’t on CRAN yet.

I don’t know if SAS makes people worse statisticians though. The numbers you “pick out” are just the ones you know to extract using functions in R. Statisticians are fond of blaming computer packages for making people “worse statisticians”, but in my experience the people they are thinking of didn’t know what they were doing anyway. For example, the use of frequency weights instead of probability weights in sample survey analyses, a mistake I have caught many people doing, arises because they just didn’t understand survey sampling to start with. Any software which gives them a choice of weights is fraught with this risk, regardless of how it handles output.

I also don’t think SAS has poor capabilities. A lot of people work very hard to make sure SAS is up to date, and for example it was well in front with GLMs, GEEs and time series analysis. It does response surface modelling, smoothing, and now it has mixed model GLMs attached. I think the problem is that the archaic interface and language are becoming increasingly cumbersome as the modern world passes them by. Andrew Gelman is probably complaining because SAS doesn’t do his part of stats (Bayesian modelling and HLMs, right?) so well, but that’s because the people it’s aimed at don’t need them. And anyway, if you are using SAS on a data set of millions, you are unlikely to be doing HLMs anyway unless you have direct access to God.

This last is uncommon in academia, but common in government and big business.

I think there is a large body of data miners, and anyone who works with statistical analysis on large datasets, who use SAS.

Also, I have been waiting for a decent implementation of GEEs in R for at least 3-5 years, so I’m not so sure it’s coming soon. It must be something that the R community don’t need. In my experience, if you want to do a decent log-linear model with clustering and/or serial dependence, SAS is the easiest way to put it together. And writing that by yourself is a lot of work. I don’t just do plug and play stats, but I am not crazy enough to pretend I can do a better job of writing a whole GEE analysis package by myself – that’s what we have smart people for!

In the latter case, are you distinguishing between the number of eyes that could have read and tested the code and the number of eyes that actually read and tested the code? As an example, the Linux kernel is freely available as source code to the entire world. Anybody and their brother can theoretically read it. Now how many people in the world do you think have read the entirety of the source code for the Linux kernel?

In addition, the quantity of people reading the source code isn’t the only factor in how many bugs are found—quality matters too. For instance, you could (theoretically) get my mother, who knows just about enough about computers to open a web browser (Sending email, though? Forget it.) to read the Linux kernel source code. Would she understand it or be able to find a bug? Unless there was a comment that said THE NEXT LINE CONTAINS A BUG, probably not.

Now I’m not saying that the first scenario you described is better than the second or vice versa. I’m just saying that the idea that more eyes automatically result in more bugs found and/or better quality code is too much of a simplification.

For various things, I use both Fortran (for libraries) and Lisp (for flexibility, metaprogramming). Of course, “older” is a bit of a misnomer in all cases, as it’s not like we’re using the original variants of any of these.

Barry @ 46: Thanks, I knew it was used in industry, but had no real feel for the userbase.

“I don’t think that’s necessarily true Walt. To the best of my knowledge the implementation of GEEs in R is pretty crappy (at least it was last time I checked) and its ARIMA functions are annoyingly incomplete. I don’t know much about the topic, but I’m not under the impression that its odbc support or sql is not at all good. ”

The advantage of open source software with a reasonably large user base is that these problems will probably be taken care of soon.

“Plus of course the help is crap, and full of nasty rude people misunderstanding your question and being rude. I think these kinds of problems come up with open source software rather a lot.”

That’s a slower problem. I expect an expanded GUI (perhaps using R Commander as a base); there are also lots and lots of books on R.

Depending on the set-up and one’s position, one might be using SAS-processed data, or SAS-generated reports, or SAS-based input screens, without knowing that SAS stands for anything other than some Brit SF guys, or a shoe company in San Antonio.

In my background, SAS is usually used as the data processing/analysis software for research projects (engineering, social science and medical). It’s convenient to use one package for a lot of stuff. The trick is data entry and validation; I’ve seen a lot of people figure that incorrectly-entered data will be ‘taken care of later’, possibly using Statistical Methodology :(