On Wed, Jul 5, 2017 at 8:34 AM, Jason Dusek <jason.dusek@xxxxxxxxx> wrote:
I can not argue these points with you; but Fortress is a good example of imperative looking code that translates to a functional/declarative core; as indeed is monadic or applicative code. LINQ is a more recent and widespread example -- though not encompassing an entire language -- of something that has an imperative form while being declarative under the hood. Scala's for comprehensions -- more or less monad comprehensions --are another.
But Linq effectively is a declarative language that's semi-SQL-like (I wish they used "project" instead of "select" but that's another question). I don't see Linq as semi-imperative.
With regards to Spark, I assume for comprehensions are an important part of the interface?
Nope. You have chained generators and you really need to watch what is parallelizable and what is not, and what is running on the partitions and what is running post-gathering/shuffling. Spark has no real facility for parallelising a comprehension.
Kind Regards,Jason
Best Wishes,
Chris Travers
Efficito: Hosted Accounting and ERP. Robust and Flexible. No vendor lock-in.
