Programmers like highly expressive programming languages, but programming managers do not. I wrote about this on Twitter a few months ago.
Q: Why do people like Lisp so much?
A: Because Lisp is so expressive.
Q: Why don’t teams use Lisp much?
A: Because Lisp is so expressive.
Q: Why do programmers complain about Java?
A: Because it’s not that expressive.
Q: Why do businesses use Java?
A: Because it’s not that expressive.
A highly expressive programming language offers lots of options. This can be a good thing. It makes programming more fun, and it can lead to better code. But it can also lead to more idiosyncratic code.
A large programming language like Perl allows developers to carve out language subsets that hardly overlap. A team member has to learn not only the parts of the language he understands and wants to use, but also all the parts that his colleagues might use. And those parts that he might accidentally use.
While Perl has maximal syntax, Lisp has minimal syntax. But Lisp is also very expressive, albeit in a different way. Lisp makes it very easy to extend the language via macros. While Perl is a big language, Lisp is an extensible language. This can also lead to each programmer practically having their own language.
With great expressiveness comes great responsibility. A team using a highly expressive language needs to develop conventions for how the language will be used in order to avoid fracturing into multiple de facto languages.
But what if you’re a team of one? Now you don’t need to be as concerned how other people use your language. You still may need to care somewhat. You want to be able to grab sample code online, and you may want to share code or ask others for help. It pays not to be entirely idiosyncratic, though you’re free to wander further from the mainstream.
Even when you’re working in a team, you still may have code that only you use. If your team is producing C# code, and you secretively use a Perl script to help you find things in the code, no one needs to know. On the other hand, there’s a tendency for personal code to become production code, and so personal tools in a team environment are tricky.
But if you’re truly working by yourself, you have great freedom in your choice of tools. This can take a long time to sort out when you leave a team environment to strike out on your own. You may labor under your previous restrictions for a while before realizing they’re no longer necessary. At the same time, you may choose to stick to your old tools, not because they’re optimal for your new situation, but because it’s not worth the effort to retool.
Related posts
(Regarding the last link, think myth as in Joseph Campbell, not myth as in Myth Busters.)
I find the source code of standard libraries quite idiosyncratic. Boost is very different from application C++, and pandas is nearly inscrutable Python inside.
If we look the problem on a language level, I do agree. If we approach the problem from a system level the things turn around. System design benefits from expressiveness, the greater the better. The problem you are pointing at is the separation of that expressiveness in layers which allow for manageable code bases. This is what your observation boils down to. A more expressive language needs that expressiveness to be used at its core. It provides a lot of freedom, but not enough structure, which forces every observer of the system to interpret aspects of that system in its own way. Such approach to a system has its benefits since each observer will accumulate unique knowledge/representation of the system. If you search for solution however(a single representation of the system), you need to synchronize those observers, and that alone requires you create another system which will be unique for every problem you apply that language to. On the other side, if you trade some of the expressiveness for structure, what you get is a set of constrains that force the observers to share aspects of the system they observe. This happens because the structure if the language creates shared nodes(both logical and physical) that are relied upon when formulating the observations. Such approach requires more work to achieve a representation of the system you are observing, because you can not jump over parts of the structure- you need to stack cumulative expressions over and over enabling higher(to lower or vice versa) dimensional transformations. Although more tedious such approach allows all observers to synchronize themselves by design- there will be deviations between the structural nodes of the language(provided you pick the same sub-network), but those are normalized at the nodes, which allows you to produce consistent result, and most importantly(as you point in your post) scalable from observer point. The big issue here is that trading expressiveness for structure, puts a cap on the system scale you can map with it(the language structure will make it impossible to observer system aspects that force the structure into partial states, and racing contexts). The latter is relevant for the other case too, but there you are forcing the limit on the observer, and the capacity of the context in which it makes the observation- as the system scales, the representation generated at the beginning will start mutating, partially invalidating itself. That partial invalidation acts as a feedback, to the observer, and will create a separate context where the data being fed back through the partially presented system fits expression wise). With each iteration that new context will slowly push the observer out of the system observed initially into a new system that actually describes the dynamics of the scaling(of the first system) in regards to the context the observer defined initially. It is a nasty drift, that is extremely consuming resource wise, because it you wish to keep the focus, you need to spawn meta-observers, but that starts pushing you towards the synchronization issue. The problem you pointed at is related to information management/modelling and not control hierarchies, although those are spawned as an answer to the general problem- you can think of them as meta-observers, that form a structure. At the end of the day programming languages are more of a side effect of a very very interesting problem. :)