r/fsharp • • Jun 12 '15

FSharp for middle size projects

http://hodzanassredin.github.io/2015/08/04/fsharp_for_middle_size_project.html
0 Upvotes

12 comments sorted by

8

u/jdh30 Jun 12 '15

Type providers

Initialy we started to use them intensively, but after some time we removed all usages from our codebase. First problem is scripts. When you created a script with a type provider and copies it to some server, first of all it shows you "The .NET SDK 4.0 or 4.5 tools could not be found". Also it increases a number of dependencies and is not very easy even if you are using Chocolately or so on. Type safety of type providers sometimes become a hell when for example you are working with json type provider and it forces you to create a lot of boilerplate code.

I've found type providers useful for exploring SQL databases but I would never use them in production code, just disposable scripts.

For example to process json value which can be null, skipped, empty or has value.

Sounds like you want schemas.

Also json type provider forces you to use F# Data: JSON Parse module but what if I prefer to use an other json parser?

And why does it have its own hand-rolled JSON parser anyway rather than using, say, JSON.NET?

Db type providers seems to be similar but all have different possibilities and problems and differnt api. It is easier to use EF with blackjack and migrations. Provaiders seems to be perfect fit for external uncontrolled data sources, but unfortunately with some drawbacks. Try to use only when you really need them.

Or when you don't plan to keep your code.

Tuples and other data structures with unnamed fields.

Initially we started to use them actively. For easier reading we used type abbreviations for every position and different type abbreviations for diferent composite types. But type abbreviation are erased at compile time and this way is too error prone.

Yes, of course. That's a silly thing to do. Read any decent tutorial book on F# and it won't do that.

For example when you inserts or removes a field in tuple or union type you are forced to fix a lot of lines where you are trying to construct or deconstruct values. Also it is very easy to do a mistake and fail at runtime when your json serializer can't (de)serialize message and throws an exception.

You're doing it wrong.

Inlining

Seems to be cool but can throw exception at runtime when used form csharp. Use only internally inside Fsharp code and mainly for performance issues.

Yes, inline is very useful.

Pattern matching and union types

Incorrectly used it moves your code and brain into the "switch case hell" which you could remember from C like languages. Currently we partially rewrites code which uses union types to simple OOP classes and interfaces.

You'll lose pattern matching, exhaustiveness, redundancy checking and the ability to add functions over that type.

Try to avoid union types creation,

Again, you're doing it wrong.

but if you really need them for tasks like AST representation then you need to be sure that your type has const number of representations.

What do you mean by "const number of representations"?

Also use only named fields for union types.

Interesting. In 8 years of professional F# programming I have never used that (new) feature.

And before writing, just think about visitor pattern again :). NEVER USE union types in your external api like rest/json. I did it once and never do that again.

You don't want to expose union types to C# code but there's nothing wrong with exposing union types to other F# code.

Monads and computation expressions

Monads is very hard to understand for novice devs and they do not compose. So try to avoid creation of your own monads and use predefined ones or from libs. Develop your own only when you really need it.

This is setting alarm bells ringing for me. You seem to be struggling with basic F# programming (e.g. pattern matching). Trying to write your own monads is trying to run before you can walk. Take it easy. Keep it simple.

Collections

Inconsistent api for different collections (fixed in F# 4) causes dramatic time loss in developing.

Elaborate?

Missing of some really useful methods like OrderByDescending (fixed in f# 4) can show you a lot of interesting stuff during debugging on production server :).

Use sortBy (~) or sortWith (fun x y -> -compare x y).

No built in generic performant solution for collections.

What does that mean?

Fsharp idiomatic libs

Unfortunately after some use we decided to switch to better tested production ready libs. They are not so idiomatic, but easier to work form both fsharp and csharp.

What libs are you referring to?

OOP support

Currently we move more and more code to OOP style from FP style. OOP feels just better and easier for maintain when your project starts growing. Usual fp project way: functions with positioned args and pattern matching -> functions with args in one record -> record with closures ->> old school OOP.

Ok. You're doing something seriously wrong.

2

u/Nishruu Jun 13 '15

Not OP, but...

Collections Inconsistent api for different collections (fixed in F# 4) causes dramatic time loss in developing.

Elaborate?

I'm pretty sure OP is referring to this, blue being existing functions, green being new additions to the API and white being intentional gaps (L - list, A - array, S - seq). I admit, it's nice to see it being standardized, but it's more of a minor annoyance (and sometimes minor performance penalty) than 'dramatic loss', IMO.

Other than that, I agree with everything else.

I have pretty much the same opinion about SQL type providers, but I have to admit I've been lately looking with interest at FSharp.Data.SqlClient, although I'm yet to test it on a 'live' app.

I'm not really anxious about writing extensive SQL queries, but I really dislike the fact that you either have to suffer extremely weak typing with a lot of boilerplate or use full-blown LINQ-compatible ORM. Usually, in C#, I use both 'full' ORM and a micro-ORM when appropriate (often within one project), but I'll keep my eye on FSharp.Data.SqlClient.

2

u/jdh30 Jun 13 '15

I'm pretty sure OP is referring to this, blue being existing functions, green being new additions to the API and white being intentional gaps (L - list, A - array, S - seq). I admit, it's nice to see it being standardized, but it's more of a minor annoyance (and sometimes minor performance penalty) than 'dramatic loss', IMO.

I'm not a big fan of that idea. For every user who benefits from the improved commonality there is another who is confused by it. For example, List.singleton x = [x] is pointless, Seq.foldBack is silly and turning O(log n) Set.contains into O(n) Array.contains, List.contains and Seq.contains is going to confuse people.

I want them to add functions that actually leverage the specific collection in question. I'd like to be able to split sets and maps in order to divide-and-conquer for parallel programming but that has been abstracted away. I'd like asymmetric union and merge of two Map collections adding the values from colliding key-value pairs but what I need to do that efficiently has been abstracted away. I'd like a heap or priority queue in Core.

2

u/Nishruu Jun 14 '15

I'm not a big fan of that idea. For every user who benefits from the improved commonality there is another who is confused by it. For example, List.singleton x = [x] is pointless, Seq.foldBack is silly and turning O(log n) Set.contains into O(n) Array.contains, List.contains and Seq.contains is going to confuse people.

I'm not sure about the confusion. I mean, it's possible, but people can be confused by a lot of things. At least in this case, the API is uniform, although you still have to know your data structures and algorithms to be able to reason about big-O.

I want them to add functions that actually leverage the specific collection in question. I'd like to be able to split sets and maps in order to divide-and-conquer for parallel programming but that has been abstracted away. I'd like asymmetric union and merge of two Map collections adding the values from colliding key-value pairs but what I need to do that efficiently has been abstracted away. I'd like a heap or priority queue in Core.

With this I agree completely. Having more 'advanced' collections out of the box, all properly tested and optimized by people smarter than me would definitely be an asset.

For anyone else reading, here's an easy example: implementing A* search means you have to implement your own priority queue (via, for example, Fibonacci heap), which is not a trivial task.

0

u/jdh30 Jun 14 '15 edited Jun 14 '15

I'm not sure about the confusion. I mean, it's possible, but people can be confused by a lot of things. At least in this case, the API is uniform, although you still have to know your data structures and algorithms to be able to reason about big-O.

Asymptotic complexity is arguably part of the API. The C++ stdlib mandates asymptotic complexities in the docs and I think that is a good thing.

In the OCaml world, people often complain that their code is slow and then learn that List.nth is O(n). This is a worse version of the same problem. Now, I agree that people should know what a list is and expect random access to be O(n) but I don't think Set.partition should have a balked function type and linear complexity when it could be logarithmic like Set.split in OCaml. Now we're going to have a million more functions that don't take advantage of the capabilities of each individual collection either, all in the name of a "uniform" API. The whole point of having different collections is that they are good at different things and that requires fundamentally different APIs. I also think these kinds of debates would be better served by a proper quantitative analysis of real code bases.

For anyone else reading, here's an easy example: implementing A* search means you have to implement your own priority queue (via, for example, Fibonacci heap), which is not a trivial task.

FWIW, here's the implementation I gave in the F# Journal article:

> open System.Collections.Generic;;

> let search (graph: IGraph<_>) src dst =
    let meta = Dictionary HashIdentity.Structural
    let closedSet = HashSet HashIdentity.Structural
    let openSet = SortedSet()
    let x = { path = [src]; g = 0.0; f = graph.distanceTo src dst; vertex = src }
    meta.Add(src, x)
    openSet.Add x |> ignore
    let rec loop() =
      if openSet.Count = 0 then None else
        let x = openSet.Min
        openSet.Remove x |> ignore
        if x.vertex = dst then Some x.path else
          closedSet.Add x.vertex |> ignore
          graph.iterNeighbors x.vertex (fun y ->
            if not(closedSet.Contains y) then
              let y'' =
                let g'' = x.g + graph.distanceTo x.vertex y
                { path = y::x.path; g = g''; f = g'' + graph.distanceTo y dst; vertex = y }
              let y' = ref y''
              if meta.TryGetValue(y, y') then
                let y' = !y'
                if openSet.Contains y' then
                  if y''.g < y'.g then
                    meta.Remove y |> ignore
                    meta.Add(y, y'')
                    openSet.Remove y' |> ignore
                    openSet.Add y'' |> ignore
                else
                  meta.Add(y, y'')
                  openSet.Add y'' |> ignore
              else
                meta.Add(y, y'')
                openSet.Add y'' |> ignore)
          loop()
    loop (), closedSet;;
val search :
  IGraph<'a> -> 'a -> 'a -> 'a list option * HashSet<'a> when 'a : comparison

Not so bad, I think.

2

u/wastaz Jun 22 '15

I'm not really anxious about writing extensive SQL queries, but I really dislike the fact that you either have to suffer extremely weak typing with a lot of boilerplate or use full-blown LINQ-compatible ORM. Usually, in C#, I use both 'full' ORM and a micro-ORM when appropriate (often within one project), but I'll keep my eye on FSharp.Data.SqlClient.

Now I'm going a bit off-topic and also doing some shameless plugging...but I've had the same feeling for a long time concerning ORM's. There's no good "middle-ground", it's either the full entity framework/nhibernate route or writing sql in strings in the code (or stored procedures). So I started a project trying to provide a more "middle-ground" ORM (or a microORM++ might be a better description I guess). It's not production ready yet, but it's getting there. Feel free to have a look, any feedback helps :) https://github.com/wastaz/Hyperboliq

2

u/kiteason Jun 20 '15 edited Jun 20 '15

Here's my take on this. I'm definitely one of the 'annoyingly vocal' community, but I'm going to be as constructive as I know how to be!

Type Providers

Missing dependencies - yes, this kind of thing is a bit of a bane when moving on from 'it works on my machine'. On the other hand you shouldn't necessarily expect to be able to copy scripts around the place and have their dependencies magically come with them. It is a bit more disappointing when you build and deploy something from -say - Visual Studio and have the dependencies not get correctly identified. That needs improving.

In contrast to jdh30, we do have some type providers in live use (albiet for reporting) and in that case they were an utter godsend.

I feel that type providers (as a concept and as implementations) have a bit more maturing to do.

Tuples and other data structures with unnamed fields

My advice on tuples is - keep them extremely local in scope. (I.e. in the context of one short function or one little pipeline of piped-together operators.) "Type abbreviations for every position" - I hadn't even thought of that but... yikes! Maybe records would be good fit here?

"It is very easy to do a mistake and fail at runtime when your json serializer can't (deserialize)"... This makes me wonder what you would want to happen when a message comes in which isn't the shape you expect. Guess at the values? Default to nulls? My feeling is that F# isn't the problem here.

Inlining

I haven't encountered this runtime exception when called from C#. Note that an important use of inline within F# code is to allow operators to be inferred as polymorphic: http://stackoverflow.com/questions/3754862/use-of-inline-in-f

Pattern and union types

This is where I disagree with you most strongly. Rather than regarding the situation as 'switch case hell' you need to think clearly about what you want your code to do in every situation. If you find this hard to do in a particular situation, it probably means your overall design, or your understanding of the domain, needs examination.

Generally people seem to agree with you on not exposing union types in APIs to be used from other languages. I haven't tried it.

Immutability

We agree!

Monads

Yeah, kind of agree. However it has been interesting for me on occasion to watch C#/OO devs painfully re-invent something like the maybe monad over a period of several days.

Equals, GetHashCode...

Agreed.

Collections

This has never really bothered me. Array.ofSeq and Seq.ofArray FTW! I strongly agree with jdh30 that it's important, once you get beyond the toddler stage, to understand what's really going on behind the scenes with arrays vs. C# lists vs. F# lists vs. sequences (etc.).

Promotion alert: my PluralSight course on 'F# Functional Data Structures' goes into a certain amount of detail on this. It isn't an advanced course - just gives the basics on the differences between the built-in collections and some hints on performance.

Fsharp idiomatic libs

I agree that there can be a maturity issue here.

Type inference

"F# does not allow type constraints involving two different type parameters". I think I'm not understanding you here. This is fine:

type MyType<'TOne, 'TTwo> ... 

So is

type MyType<'TOne, 'TTwo when 'TOne : equality and 'TTwo : null> ...

As I say I think I'm misunderstanding

OOP support

Sorry, I agree with jdh30 on this one. It feels like something is going wrong culturally.

Edit - various typos.

2

u/acmd Jun 12 '15

Well, that's unfortunate, however not unheard of: functional structural elements work best at small scale level.

4

u/PM_ME_UR_OBSIDIAN Jun 12 '15

It really, really depends. My working theory is that functional elements work better in the data pipeline, while OOP/actor/reactive work better for state machines.

Reading through the lines, I don't see a criticism of F#, I see a criticism of the design pattern literature in functional programming. I've had few of the issues mentioned in a large real-world product, because there was a strong oral tradition on the team of which features to use where.

1

u/sixbrx Jun 13 '15 edited Jun 13 '15

Biggest things pushing me towards OO/classes in my F# code is the ability to define custom constructors for classes, where data is validated and invalid data can be stopped "in its tracks" as early as possible by throwing an exception, before it corrupts who knows what downstream.

I've found that even really "simple" types can often be seen in retrospect to need data validation at construction time, so there's no clear dividing line as to when a full blown class is necessary, making it safer (time/effort-wise) to use classes from the start.

I wish it were not so, it seems like such an easy thing to fix. Luckily F# is still brilliant for programming with classes as well.

2

u/catlion Jun 19 '15

Make illegal states impossible(c). Validation applies to the objects interacting with the"outside world" for sure, but not the core"domain" models, imho.

1

u/kiteason Jun 23 '15

Another thought - on this point:

Monads ... do not compose.

I'd be interested to know what OP means by this. In a sense monads are the essence of composition. Here's a great example of computation expressions (which overlap with monads) composing:

http://www.m-brace.net/programming-model.html