r/scala • • Aug 21 '17

Functional Programming and Modern DevOps: Aaron Contorer. With scathing critique of scala in production environment.

https://youtu.be/ybSBCVhVWs8
3 Upvotes

67 comments sorted by

17

u/ratherbefuddled Aug 21 '17

Aaron is founder and CEO of FP Complete, the software technology firm commercializing the Haskell language for Functional Programming.

from LinkedIn

He's entitled to his views of course but one might be forgiven for wondering how objective they are.

IMO Scala's problems don't stem from allowing immutability because in practise it's used sparingly. The main problems are weak tool support, compiler speed, and some hangover from early implementation choices which are in hindsight regrettable. All of this is improving, but I'd like it to be faster.

From a devops point of view the scala world is far easier to deal with than many others. Try dipping your toes into the node.js ecosystem if you want to feel better about it!

3

u/vagif Aug 21 '17 edited Aug 21 '17

His critique though are not about immutability. He specifically mentions cryptic unreadable syntax of scala. Where he says that coming back to the scala code you wrote yourself you cannot be sure anymore what it means.

16

u/Jasper-M Aug 21 '17

Show a piece of Haskell code written by a category theory expert to someone only vaguely familiar with Haskell and ask them how cryptic and unreadable they find it...

3

u/vagif Aug 21 '17

But we are not talking about scala code written by category theory expert. We are talking about scala code written by normal "enterprise" developers.

I use haskell in production for more than 5 years already, and no one in my team ever had a problem reading any code in our projects.

I cannot say the same thing about scala.

11

u/denisrosset Aug 22 '17

Why post Scala hate in the Scala subreddit?

I did not have any problems with readability of recent Scala projects. That includes algebra, cats, circe, spire where I made contributions. My only current pain point is SBT, but that's more due to the complexity of the modelling (and of the underlying domain) than the few symbolic operators still used.

And in the last 4 years, I was able to go from imperative programming to understanding pure FP style in Scala, to the point I understand how to write equivalent code in Haskell. Whereas I got stuck writing Haskell code before, because the programming model was so foreign.

7

u/aloiscochard Aug 23 '17

Where else should such post be posted?

Criticisms are pretty useful to improve a given technology and know how a given public feels about it, why rejecting entirely negative criticisms?

8

u/Milyardo Aug 21 '17 edited Aug 21 '17

Just because he says it, doesn't make it a critique. The cryptic syntax adage gets repeated ad nauseam because it's thought terminating. It can't be demonstrated concretely, no two persons will agree ever to what constitutes as an example of cryptic code.

There's a similar situation with the C++ comparison, it vaguely implies as if there's some merit to reductivist programming languages, which is hypocritical if you're simultaneously advocating for any modern strongly typed programming language. But it gets repeated because the argument is thought terminating because no one agrees what baggage C++ carries from C holds the language back , or if C++ ruined C with extra features.

EDIT: The C++ argument is also further meritless because it attempts to relate Scala to Java as if Scala is a superset of Java like C++ is a superset C, instead of it being an entirely different language.

8

u/KagakuNinja Aug 21 '17

IMO, Scala is definitely the C++ of modern times. Not that it is a superset of Java, but that it took the dominant language ecosystem of the times, and bolted a trendy paradigm onto it. Scala is a gateway drug for imperative programmers to dabble in FP; like C++ enabled C programmers to do OOP.

Like C++, Scala is hated by the old guard, who fear and dislike the new features ("Too slow! Complex! How will we find people who know this new language?").

It is also hated by the FP purists, because it isn't pure enough, just like the Smalltalk people hated C++ and Java.

7

u/mdedetrich Aug 21 '17

IMO, Scala is definitely the C++ of modern times. Not that it is a superset of Java, but that it took the dominant language ecosystem of the times, and bolted a trendy paradigm onto it. Scala is a gateway drug for imperative programmers to dabble in FP; like C++ enabled C programmers to do OOP.

Except that this isn't true, and has been stated by the language designers many times. Scala isn't a true superset of Java (i.e. constructor overloading is more limited in Scala), and a lot of the design decision in Scala are deliberate (i.e. Scala does deliberately have subtyping because its the only proper way to model modules in a language)

2

u/KagakuNinja Aug 21 '17

Didn't I say "Not that it is a superset of Java"?

The point of the language was to make a hybrid OO/FP language.

Regardless of the "goals", Scala is comfortable for Java programmers, because we can drop back into imperative style any time we want. This makes it a good transition language for imperative programmers that want to learn FP.

4

u/valenterry Aug 22 '17

Didn't I say "Not that it is a superset of Java"?

But you said it "bolted a trendy paradigm onto it". I don't think you can say that. It implies that Java would be some kind of "base" where something was put on. But Java is not the base. The same runtime is (or was) but that is a different thing.

Just stick with OO/FP which is true, but better not use Java for that argument.

3

u/mdedetrich Aug 22 '17

Exactly. Kotlin is an example of a language that has a "bolted a trendy paradigm onto it" (with it referring to Java). Typescript is another example from Javascript land

-2

u/vagif Aug 22 '17

But the said "purist" recommended in his talk Clojure and F#, so your argument does not hold.

In fact as a haskell dev, I love working with clojure despite of it being both impure and dynamic, yet I hate scala with passion.

It is over-engineered, overly complex, and allows easily producing very obfuscated code.

7

u/mdedetrich Aug 22 '17 edited Aug 22 '17

Haskell has code which I consider some of the hardest that I have ever read, only really complex memory management code in C++ holds that title as well. The C++ is the only example which I think should be excused from having hard to read code seeing as doing memory management is incredibly hard problem to do.

The very frequent overuse of both operators and terminology which only people that are very familiar with category theory can understand plus the fact that most Haskell libraries aren't properly documented (their documentation is just auto generated from code which isn't good enough) contributes to this.

Clojure is quite easy to read, but it has all of the same problems that Scala has due to Java interopt (and Scala is actually trying to remedy this, i.e. see scala-native).

Not trying to dismiss Scala's issue, it definitely has them, but don't go around throwing rocks in a glass house.

10

u/ratherbefuddled Aug 21 '17 edited Aug 21 '17

His critique though are not about immutability. He specifically mentions cryptic unreadable syntax of scala. Where he says that coming back to the scala code you wrote yourself you cannot be sure anymore what it means.

His critique is focused on mutability and side affects, he mentions syntax only in passing. He describes Scala as to Java as C++ to C, and says it's a great language to write bugs in in the same way.

The syntax criticism is old and not really relevant these days, newer versions of libraries and tools have made strides here.

In reality the mutability concern is smaller than he makes it out to be, and is balanced by both the advantages of the JVM and java ecosystem and the type system, neither of which are really mentioned. I don't think it's a very informed view of Scala but the purpose is to promote pure FP in particular Haskell so that's not too surprising.

1

u/aphexairlines Aug 30 '17

Scalastyle can do away with cryptic names:

<check enabled="true" class="org.scalastyle.scalariform.ClassNamesChecker" level="error">
  <parameters>
    <parameter name="regex">^[A-Z][A-Za-z]*$</parameter>
  </parameters>
</check>
<check enabled="true" class="org.scalastyle.scalariform.FieldNamesChecker" level="error">
  <parameters>
    <parameter name="regex">^[A-Za-z][A-Za-z0-9_]*$</parameter>
  </parameters>
</check>
<check enabled="true" class="org.scalastyle.scalariform.MethodNamesChecker" level="error">
  <parameters>
    <parameter name="regex">^[a-z][A-Za-z0-9]*$</parameter>
    <parameter name="ignoreOverride">true</parameter>
  </parameters>
</check>
<check enabled="true" class="org.scalastyle.scalariform.ObjectNamesChecker" level="error">
  <parameters>
    <parameter name="regex">^[A-Z][A-Za-z]*$</parameter>
  </parameters>
</check>

14

u/mdedetrich Aug 21 '17

His critiques make zero sense, I mean sure there its possible to do mutation of state in Scala, but....

  • Clojure and Ocaml (which he mentions often as recommended languages) are just as bad as Scala in this regard

  • Nothing is stopping you from mutating state in Haskell either, and their FFI + common linking with C poses the same problems that Scala has when linking with Java

  • Clojure also has seamless interopt with Java which poses the same problems that Scala has

  • Clojure is even weaker than Scala in this regard (although both languages use immutable collections as a default)

  • Typical Scala code minimizes side effects, and if you do PFP then even these side effects are encapsulated (see cats/scalaz)

None of his commentary on Scala appears to be rational compared to the rest of his talk, it sounds like Scala kidnapped one of his kids.

0

u/vagif Aug 21 '17

I think the confusion here is that he critiques scala right after talking about mutability, so the impression is that he blames mutability in scala.

But it is not the case. His main scala critique is about cryptic language, that has nothing to do with mutability. That's why he recommends clojure and F# as a much more readable and simpler languages. Mutability in this case does not play any role.

7

u/mdedetrich Aug 22 '17 edited Aug 22 '17

Except that this isn't true, its one of his criticisms but his main ones are actually about mutability and purity. In his talk he emphasis purity (i.e. side effect free functions) ad naseum, yet the other language examples he uses (barring Haskell) are worse in this regard compared to Scala

1

u/jackcviers Aug 23 '17

I love scala, and I love types. I think they make for awesome documentation and prove a program works.

Types do clutter up code:

    fmap' f (Just x) = Just (f x)

    fmap' f Nothing = Nothing

is easier to read than

    def fmap [A,B](fa: Option[A])(f: A => B) =
  
      if(fa.isEmpty) {
    
        Option.empty[B] 
  
      } else Option(f(fa.get))

because of type inference.

2

u/m50d Aug 23 '17

That's not really a fair comparison though; you can write the Scala very similarly to the Haskell:

def fmap[A, B](f: A => B): Option[A] => Option[B] = {
  case Some(x) => Some(f(x))
  case None => None
}

4

u/jackcviers Aug 23 '17

Oh, the pattern match wasn't what I was referring to - it is the necessity to declare the generic A and B and A => B. Scala cannot infer A => B. That is, I cannot write:

     val fmap = fa => f => fa match {
       case Some(x) => Option(f(x))
       case Nothing => None
     }

or:

    def fmap(fa)(f) = fa match {
       case Some(x) => Option(f(x))
       case Nothing => None
     }

.

The problem is that it cannot infer a generic return type correctly. It cannot infer the function type in the argument list.

The part of scala that makes people cry rocket science, is the generic parameters.

2

u/theQuatcon Aug 23 '17

FWIW, this has been my experience as well.

I have no problem writing monad transformer code in Haskell, but the equivalent code in Scala is just so incredibly littered with syntactic noise (such as type ascriptions, generic parameter declarations, etc.) that I usually just give up.

(Type inference is a hard problem when you have full subtyping, so I don't think it's unreasonable to require these ascriptions, but I do think that the language suffers a lot as a result. I'd rather do without subtyping, but I guess that's why I'm using Haskell rather than Scala whenever I have the choice.)

1

u/jackcviers Aug 23 '17 edited Aug 23 '17

The subtyping makes interacting with java libraries buttery smooth. Which is a feature.

There's not a really good tutorial on using scala's generics, outside of the shapeless tutorials. There needs to be a basic generic type tutorial.

Something on the lines of:

Scala uses generic types. This is a generic F:

F[A]

Generics mean a type that can hold some other type. Generics can hold more than one type:

F[A, B]
G[A, B, C]

The convention is to name types contained by generics with single character uppercase ASCII characters starting from A within a given scope.

Types in a generic are not concrete. They don't represent any real type. An A could be an Int ... etc.

There needs to be a type cheatsheet in the sidebar.

Now that I understand the type system -- I love the types now, and they don't look like noise. But it takes a long time to piece together the understanding from experience of using the language.

2

u/theQuatcon Aug 23 '17

Well, sure, but I'm actually not all that interested in Java interop, though I'm not sure Scala would have any adoption at all without that interop. It's a tough balance/question for any language designer. Personally, I'm mainly using Scala for the JVM. (But then again, I'm not even sold on that based on technical merit... it's just that you don't have to make the business case from scratch when you can just say "JVM" and all the non-technical people just assume...)

Of course, given the abysmal binary compat story for Scala, I'll actually consciously use Java libraries in preference to Scala libraries. (I'm moderately hopeful that TASTY will help a lot here, but it's going to be years and years...)

1

u/jackcviers Aug 23 '17

There's no real way to do files without java interop, or write to sockets, or log to console, or work with streaming buffers. There's no from scratch http stack. You can't work with parallelism without it.

If scala had a story for these things, that would be great, but it doesn't.

The JVM is fantastic. It really is. Real cross-platform compatibility is necessary. But the jvm isn't the only way of achieving that.

I use Scala because it's the best FP programming language to be employed, and has the least amount of development headaches.

2

u/theQuatcon Aug 23 '17

There's no real way to do files without java interop, or write to sockets, or log to console, or work with streaming buffers. There's no from scratch http stack. You can't work with parallelism without it.

Uh, what? Of course you can -- there's no need to import the semantics of the underlying platform into your language. Hence why anything other than CPU microcode works. And hence why "IO" in Haskell works. (I mean, it's not particularly elegant and would probably be designed differently given the benefit of hindsight, but there's no fundamental problem here.)

I use Scala because it's the best FP programming language to be employed, and has the least amount of development headaches.

I'm happy that you're happy with it, but it's caused endless completely unnecessary headaches for me, esp. given the terrible depencency situation. (But, again, as I say, I'm mildly optimistic that it'll eventually be fixed. Btw, most of this problem could have been fixed by just mandating the shipping of source code like Hackage does. Then again: Would any "enterprise" go for it? Probably not.)

→ More replies (0)

1

u/mdedetrich Aug 24 '17

Scale doesn't have subtyping due to Java interopt, it has subtyping because it's the only way to have proper modules in your language, which Haskell is severely lacking

Also while you may see types as noise, they are tremendously useful when it comes to documenting and readability of functions

1

u/theQuatcon Aug 24 '17 edited Aug 24 '17

Nobody's preventing one from declaring types for functions at the top-level (and it is in fact considered Good Style in Haskell). I'm talking about declaring types for local/helper functions -- it's incredibly noisy and mostly pointless.

EDIT: Re: modules and subtyping (and I can't tell you how weird it feels to say this about !Haskell): IME this is largely a theoretical concern and has absolutely minimal impact in practice. If there were a dedicated "module language" (like in e.g. ML) then it might be a more valuable thing just because it would be less prone to abuse, but as it is, I'm mostly just seeing inheritance being abused :(.

(And the cake pattern which was touted as The Solution To Modularity(TM) is rightly seen as 'misguided' these days.)

→ More replies (0)

1

u/jackcviers Aug 24 '17 edited Aug 24 '17

I don't see them as noise. I see them as extremely useful. I've gone through the trough of understanding though.

That trough is deep and wide at first, but out made me understand, even in other strongly -typed languages, how I made many assumptions about the types of my code.

The more I think about modules, I'm not clear on what that term actually means, and how subtyping is required to do it. Can you clarify?

EDIT: Most of the time, people are speaking of traits when they speak of module systems. I don't think you need traits, specifically, for a module system -- https://typelevel.org/blog/2016/09/30/subtype-typeclasses.html

You can do it with implicits implicit classes and typeclasses. You don't really need extends, but it does involve an implicit wrapper class to access the this.functor.map as this.map. I may actually try writing a full monad heirarchy to see if it works later.

5

u/vagif Aug 21 '17

The Scala comments start at 17:45.

3

u/Milyardo Aug 21 '17

I skipped through it, he mentions tools, talks vaguely about functional programming, a few slides mention Haskell and Clojure. Didn't catch where he specifically talks about Scala anywhere.

1

u/vagif Aug 21 '17

Starts at 17:45

9

u/Milyardo Aug 21 '17

I get the impression for some reason he hasn't actually worked with Scala(almost certainly not within the last half decade) and is working off opinions he heard second-hand.

2

u/jdh30 Aug 21 '17

Does he have any industrial experience?

5

u/TensorMetric Aug 22 '17 edited Aug 22 '17

He praises Haskell as the best FPL for productivity, performance and quality and is misinformed about why F# isn't popular. Also calling Scala just a better Java and comparing it to C++ is disingenuous.

https://youtu.be/ybSBCVhVWs8?t=952

5

u/jdh30 Aug 22 '17 edited Aug 22 '17

I cannot believe he says Haskell is fast. He also says "Haskell makes fast native code as opposed to JVM or .NET code". What a load of hogwash.

"Sorry for OCaml that it hasn't been adopted more". I'm still seeing far more OCaml in industry than Haskell.

This guy is a former program manager with a degree in cognitive psychology. What he lacks in technical knowledge he makes up for in marketing. Specifically, speaking about Haskell at 7CTOs and advising them to embrace Haskell is a great way to drive adoption.

8

u/ivanovich_ivan Aug 22 '17

Its funny that this article has 70 upvotes in the haskell sub-reddit and not so much in /r/programming /r/fsharp or even here.

I don't really understand the point of these functional purists. To me scala is the real sweetspot. Better than Java considering just the OOP part. Has good/enough functional support for one to write functional code. Maybe its not the best language for pure FP, but my question is pure FP worth all the trouble? Aren't some programming problems naturally suited to imperative/OOP style and some to functional style. This is the very idea on how scala was built.

10

u/m50d Aug 23 '17

Maybe its not the best language for pure FP, but my question is pure FP worth all the trouble? Aren't some programming problems naturally suited to imperative/OOP style and some to functional style. This is the very idea on how scala was built.

I've found that over ~7 years of professional Scala I've become more FP-purist, solely based on experience of what goes wrong when you violate purity, to the point that I consider many of Scala's less-pure features to be undesirable and ban them from my code. I would probably never have got started in Scala without them though. That's where Scala has succeeded for me: it offers something like Haskell but with training wheels you can use to stay productive as you're learning better ways to do things.

1

u/woztzy Aug 24 '17

I agree. (I also prefer Scala's strictness, tooling, and sane dependency management and available libraries, but that's beside the point.)

"Better ways to do things" is a rabbit hole.

Something to be familiar with, along with data structures and algorithms, upstream library and platform knowledge, and various levels of domain knowledge.

This is why I can understand the appeal of languages like Go and Python, that are easy for beginners to get things working in, but don't necessarily make it easy, especially when trying to write correct and extensible systems. It would be nice if functional Scala were as approachable, without giving up the advantages of purity or resorting to using ungoogleable synonyms for existing patterns.

7

u/Milyardo Aug 22 '17

I'm pretty sure it's not getting upvotes here in Scala because of the bait-and-switch clickbait title. Had it been titled to be a video about functional programming, and not specifically a critique of Scala where non exists it would be better received.

6

u/jackcviers Aug 22 '17 edited Aug 22 '17

Is pure FP worth the trouble?

The problem with any undeclared side-effects or escape hatches is that just one side-effect allowed is enough to undo any referential transparency in a pure program.

One throw can make it impossible to compose one function with another. One file read or write can also cause this.

In order to accommodate escape hatches the abstraction (functions) has to be abandoned. This raises the unit of abstraction to something else - a module, a class, a type, a package, all the way to possibly an entire program.

Functions are simple objects - they have a limited number of argument's, the argument's have static, reified types, all the argument's are non-null, all the argument's are immutable, all types are present in the type signature, possibly all capabilities for all types in the type signature are declared in the type signature. This means you have all the information necessary to understand a function body at the site of declaration. You don't need anything else.

Obviously, if you make the unit of abstraction larger by using side effects, you now must understand all the functions in the module in order to understand the fundamental unit of abstraction. This raises the minimum number of parts to understand by at least one (the side effect has to go somewhere). It is more complex because it has more parts.

Complexity is what causes programming errors. Errors cause productivity to fail. Eventually, programs with side effects become so complex that you have to be an expert in the program to understand it, rather than an expert programmer.

FP concepts keep interfaces small. They manage the complexity of the program by trading off more complex function interfaces for less complex overall programs. They trade off complex composition (you must map to unwrap and transform my value by passing a function to map) and names for smaller groups of abstractions. They trade encapsulation for declarative interfaces. They use mathematical names because it makes it easy to Google them. There's lots of outside help available to understand programs that are lawful and functional. It's all in the types.

OOP aims to hide and encapsulate complexity to manage it. However, you can never truly know what code you are calling is doing without jumping into it at runtime. This makes it more difficult to compose OOP programs than functional programs. You have to be an expert on a larger volume of the code. There's a little outside help available outside of some patterns books. There are no certain laws to follow.

Imperative programming makes no attempt at managing it - that's the job of the programmer. You have to be an expert in the program - there is very little outside help available.

EDIT: Forgot to type my conclusion:

Conclusion: If programmers want to manage complexity, then pure FP is the simplest (has fewest parts), paradigm for doing so. If managing complexity is the goal, it is the only paradigm that solves the problem in any sort of verifiable way.

1

u/mdedetrich Aug 22 '17

Where is the link to the Haskell subreddit?

2

u/ivanovich_ivan Aug 22 '17

Check other discussions on top just below the scala logo.

1

u/pipocaQuemada Aug 24 '17

Maybe its not the best language for pure FP, but my question is pure FP worth all the trouble? Aren't some programming problems naturally suited to imperative/OOP style and some to functional style.

Pure FP languages, surprisingly, don't force you to program everything in a functional style, just a pure one. It's not especially difficult to write imperative code in Haskell.

6

u/mukhsw Aug 23 '17

With all regrets, I must say the man fears Scala the most. He mentions Clojure on JVM, F#, OCAML as good choices on existing widely deployed platforms like JVM, .NET, etc. All these languages are impure. So why not Scala. Because with libraries like Scalaz (the upcoming Scalaz 8), Shapeless, Monix, Cats and many others its is possible to do principled and pure functional programming in Scala. You just have to look at the Scala ecosystem. Inspite on being JVM, fabulous range of libraries have been written many of which are inspired by Haskell. BTW, I use more Clojure than Scala. But as a FP language supporting advanced category theory concepts it is one of the best, dare I say the only practical one. Haskell, Clojure, Scala, Purescript, Erlang, F#, etc. are all worthy contenders. I would prefer Scala because:

  • You can deploy on most widely used industrial platform of today i.e. JVM. Create a Jar and you are good to go in Elastic beanstalk or wherever.

  • Use the benefits of FP (concise code, safe code, multi-core ready and may other things). After all business need to ship software fast and one that scales (Scala is right up there).

  • Use huge amount of libs to get started and then build great FP wrappers on them (which already is being done). And as for the developer:

  1. It is on one of the best ecosystem - JVM (think all the libs)

  2. All Haskell kind of abstractions are there in so many libs in Scala (Haskell guy can get his intellectual fill and make money)

  3. It has compiler macros (Lisp guy would not be so depressed or maybe. I am not saying Scala is homoiconic :-)

  4. You have Erlang style Actor implementation (and competing ones too. So something for Erlang guys though JVM and BEAM are different creatures). Hey we have got JVM, Haskell, Lisp, Erlang, ML style covered on an industrial platform (Oh by the way I forgot ScalaJS - Browser, nodejs, React Native, etc.). The above video is by a man whose company promotes Haskell. And he is such a fud propagator. CTOs! (BTW, I am a CTO too), please ignore this man. Haskell has many top ambassadors and you do not need such a guy from FPComplete. Lastly, give Scala a try (after all top companies with some of the most serious talent in software world are on it). Let us all coexist and to each what suits him most. Peace.

9

u/corn_dog Aug 21 '17

Haskell consultant says scala isn't very typesafe. No surprise there.

4

u/azzie_elbab Aug 22 '17 edited Aug 23 '17

The reason he singled out Scala is that Scala(for all its imperfections) is big enough in FP community to actually compete with Haskell in that particular niche

7

u/KagakuNinja Aug 21 '17

He doesn't criticize Scala in a production environment, he simply claims that Scala is a terrible language because it allows mutable state and side effects, in order to be compatible with Java. He then mentions offhand that it is OK, if you just want something better than Java... It is the same old functional purity argument.

That said, it was an interesting talk.

10

u/marcinzh Aug 21 '17

OCaml and F# are exactly as impure as Scala, but he gives them a pass.

3

u/mdedetrich Aug 22 '17

Considering the ecosystem, they are actually even more impure in Scala. OCaml actually advertises the fact that they don't use monads as persuasively as Haskell, and F# has limitations in this regard because they don't have HKT's (due to having to interopt with the CLR)

1

u/[deleted] Aug 25 '17

He just kept talking about how happy everyone is in Haskell and how they are way more productive. Andddddd I'm gonna try it and report back here if it is BS