Software has two ingredients: opinions and logic (=programming). The second ingredient is rare and is typically replaced by the first.
I blog about code correctness, maintainability, testability, and functional programming.
This blog does not represent views or opinions of my employer.

Sunday, August 20, 2017

Comprehensive real life functional programming examples



Several years ago, I think at Uberconf, I was listening to yet another talk that showed some list manipulations and called it functional programming.  I also remember being very confused about how to use FP in real life.  Seeing bunch of examples did not translate, for me, to being able to write end-to-end application using these concepts.  I remember talking to my colleague that it would be really good to have a comprehensive example, not just isolated 5 code-liners.


About a year ago, I decided to just do it.  I decided to work on a simple CRUD application and implement it in a fully functional way.  The idea grew and became:  Polyglot application made out of interchangeable front-ends and back-ends implemented using various languages and functional libraries.  https://github.com/rpeszek/typesafe-web-polyglot. At this moment I have 3 back-ends (Yesod-Haskell, Servant-Haskell, Http4s-Scala) and 2 front-ends (Elm, Halogen-PureScript).   I will write short blogs describing my goals behind each of these projects.  This post describes my general goals.


Comprehensive, what do I mean?

These are relatively small apps (a bit over 1000 lines each) that still facilitate the amount of code reuse I would expect to see in a large application.  There is quite a bit of similarity between domain objects that expose create-retrieve-update-delete interface,  thus, I expect to have code reuse too. These apps do what typical web-apps do today: talk to database, do Ajax communication, deal with Json, etc.  
This is just about CRUD so there is no business logic to speak of and things are rather dull, but I think it becomes clear how would adding business logic look like in that code.


What is still not implemented?

Security,  better styling (completely ignored in my current PureScript app), more complex (has-many) relationships.


Fully functional way, what do I mean?

Referential transparency (same computation, same inputs == same results) -  this ought to be the defining property of FP since it enables everything else.  Effectful computations can still be be referentially transparent.  All of my apps treat this aspect with due respect. For example, my Scala backend app does only one unsafe operation:  creates in-memory H2 DB at startup.  
(Side-note: sadly this aspect did not make the cut in design of Java 8 streams - I say streams are based on opinions, not programming).


Reasoning about code -  I find that the use of type class polymorphism retains good correspondence to logic.  Method signatures look like rotated Gentzen notation with clearly identifiable assumptions (type constraints) and conclusion (result type).  Here are 2 examples:

PureScript:
type AppM eff = (Aff (ajax :: AJAXappconf:: APPCONFIGnav:: NAVIGATION | eff))
ui :: forall eff model.
                 Show model =>
                 EntityRoute model =>
                 EntityReadHTML model =>
                 EntityGET eff model =>
                 Proxy model -> H.Component HH.HTML Query Input Void (AppM eff)

Scala:
case class CrudHandler[K,D,E[_]](uriString)(implicit evMMonad[E]
                                              evPersitAsTaskE ~> Task,
                                              evConvertKeyIntId[K],
                                              evPersistPersistCrud[K,D,E],
                                              evJsonKEncoder[K],
                                              evJsonDEncoder[D],
                                              evDecodeJsonDDecoder[D]) { ... }

Even without knowing what these are or wanting to admit that logic is important in programming it is evident that having the information organized into logical assumptions and conclusions makes things clear. For example (PureScript), I know that I am only using GET REST effects (no PUT, DELETE, etc), I know that my code is NOT using == comparison or has no sorting logic...


Polymorphism and generalization - most of my examples try to be polymorphic in both domain objects (being CRUD-ed) and in effects used.  For example, the same code works with database back-ends and STM persistence and, at the same time, works across domain objects.  In OO world that approach would probably violate KISS.  In FP generalization is essential tool because it limits solution space (parametricity) and makes it harder to write programs that compile and do not work.
(Some projects have ‘simple’ branch which is more ‘hard-coded’ and less polymorphic).


Type safety - I am interested in type safe FP (hence the selection of language choices).  All application provide type safety over effects. Unique types are used as much as possible (for example integer IDs used in URLs or as DB keys have unique types instead of using just Integer. Big advantage of using typed functional languages is type safety in deconstruction (pattern matching) often referred to as non-exhaustive pattern match error. This type safety feature comes for free (maybe except for Scala which needs sealed annotation) and my example apps, obviously, take advantage of it.
Just looking at the method signatures shown above, one has to a least entertain possibility that this amount of type information could imply stronger type checking.


Composability - standard monadic and applicative approaches are used to achieve composability. I think real-world examples like these CRUD apps make monadic computations feel right at home and feel very readable.  
Side-note: It was somewhat of a revelation for me when I finally realized (Eugenio Moggi∗original https://core.ac.uk/download/pdf/21173011.pdf) that sequential composability == monads.  Basically, composability == category,  composable computations == Kleisli. You want composable code you need monads.  Maybe trivial, but sends shivers down my spine.



Saturday, July 29, 2017

Scala what’s wrong with you!

I want to share this as, hopefully, a somewhat unique take on Scala by a complete Scala’s newb. My team at work is evaluating new technology choices and I was tasked with evaluating Scala.  

A little about me:  I program mostly Groovy/Grails at work and Haskell at home.  Have been coding Haskell for a bit over 5 years and Groovy/Grails for maybe a year longer.  I started as a big enthusiast of OO (long time ago), now I enjoy different things (like TAPL book).  I am disclosing this because words like important, practical, right, and wrong tend to change meaning depending on the background.

To make my transition to Scala easier,  I decided to use Scalaz. I ended up using both Scalaz and Scala std library side-by-side. That let to somewhat shocking realizations:

val ints: List[Int] = List(1,2,3)
val strings: List[String] = List(“hi”, “there”)
val test: List[Int] = ints diff strings

That will never ever subtract anything from the list but compiles just fine and runs without error.  Just switch to scalaz IList and nonsense like this disappears.  This is disappointing. Code like this is the reason why I want to run away from Groovy.  Even C# would not allow such code to compile (C# happens to be the other technology choice evaluated by my group).

Type inference

I get that figuring out a decent type reconstruction algorithm for a language with subtyping was a seriously impressive achievement.  But what is the practicality of it?  To make programming more error prone?  Here are some examples:

val test  = "a" :: List(1,2,3)                       
  res: List[Any] = List(a, 1, 2, 3)
val test = List(1,2,3) ++ List("a","b")              
  res:  List[Any] = List(1, 2, 3, a, b)

The pragmatic take on this is:  if the inferred upper bound is Scala’s Any a 99% chance this is a program bug and a 1% chance that this is what programmer intended (and I am generous here). I find this really concerning.  Refactoring will introduce bugs, think about changing 1-1 relation to 1-many.  

You can switch to using scalaz where type inference appears to be magically safe:

val test:  = "a" :: IList(1,2,3)           
  error: type mismatch
   found   : String
   required: Int
        "a" :: IList(1,2,3)
        ^
val test  = IList(1,2,3) ++ IList("a","b")   
  error: type mismatch
        ...

Googling on the subject tells me that type safety concerns like these seem to be answered as ‘just use invariant types’. Invariant types help but:

case class TwoOfTheSame[A](x: A, y: A)
val test = TwoOfTheSame(1,"hello")  //TwoOfTheSame[Any] = TwoOfTheSame(1,hello)               

Equals

In a language where people can (and do) write code like:

list.map(_._2).filter(_._3.id == someId)  

Lack of type safety on == is just crazy.  Quoting Paul Phillips (youtube) Java has jumped of a bridge and Scala followed.  Incidentally, just use scalaz === and you will be fine.

Type safety ... add more types   

Adding more types makes things better, at least that ought to be the conclusion from Mars orbiter crash.   As an example, database primary keys tend to be Ints or Longs.  I could use distinct types instead!  Unfortunately, that will just exacerbate the type safety problem with equals:

case class DepartmentPkey(id: Long)
case class EmployeePkey(id: Long)
case class Employee(id: EmployeePkey, name: String, deptId: DepartmentPkey, ...)
val deptId:  DepartmentPkey = …
employees : List[Employee] = ..


//all of these compile
employees.filter(_.id == deptId)           
employees.filter(_.id.id == deptId)       
employees.filter(_.deptId.id == deptId)
employees.contains(depId)

These kinds of issues is what makes some of us very upset (see the above Paul Phillips talk youtube, and also this scalaz ticket).  Others do not see these as a problem.  People are just so used to achieving correctness by hours of staring at the code, debugging, and testing that most of us do not recognize these as a problem.  Correctness by effort is the status quo.

So is Scalaz the only good thing about Scala?  

No there is also Cats library.  Just kidding!  I think Scala has lot of good things going for it.  It is the only language option that allows for slow migration path from OO to FP.  It has community that mostly tries to work together across the OO-FP divide and seems to benefit from this diversity. (See this infoq video.)  
Has DOT calculus that took 8 years of research (youtube) making Scala one of the very few languages that recognize the importance of solid theoretical foundation.  Has new dotty compiler project that should result in more maintainable, sane, and much faster compiler.  Has plans for language supported effects!  (dotty shows Effect in the feature list with status ‘considered’).

In my opinion:  

All programming languages (possibly with exceptions of lambda calculi and such) create this closed, limiting environment of thought.  Programming is thinking inside the box.  Compared to mainstream alternatives Scala make it a larger box.

Scala is this amazing place where OO and FP can be placed side-by-side for an eye opening comparison.  There is a need for a library that is more approachable than Scalaz or Cats to champion this. The assumption has to be made that people will not invest time to learn FP unless they see immediate benefits and the learning curve is minimal.  It is sad that the standard library lacks basic type safety to play this role.

I am very excited about the possibility of language level support for effects.  This will introduce effects into mainstream and other languages will eventually follow.  Effects may finally be treated with due diligence!  I think about software engineering as a predator with logic as the pray.  It is important not to kill for the sake of killing (think mutable variables), kill only for food (think updating DB record).  Programming and logic can live happily together and if you do it right they are the same species.

Scala was designed with OO engineer in mind.  It is a big compromise for functional programmers (see scalaz ticket linked above).  FP-ers choose to code in Scala because it is the only (established) choice on JVM that has things they need (like type classes or higher kinded types).
Scala is also overly complex.  Things can be made much simpler if OO is dropped from the table.  Solutions like the Eta project could prove a better choice for bridging OO and FP on the JVM.  The cost of that bridge is in a more structured FFI and not in the complexity of the language and compiler.  Bridging OO and FP can happen using two languages, not one.  This option will make functional programmers happy.  

I have done two months of Scala.  I have worked on 3 small projects using it.  One of them is on my github.  I am very impressed with the quality of of the libraries I have used: http4s, sql library with funny name: doobie, scalaSTM (I have implemented a minimal set of scalaz bindings for it), shapeless, circe, scalacheck, Scalatags.  It was quite inspiring experience after Groovy.  Scala can write really nice code.

Thank you for reading.