= Proposed Self Contained Change: Apache Spark = https://fedoraproject.org/wiki/Changes/ApacheSpark Change owner(s): William Benton <willb@xxxxxxxxxx> Apache Spark is a fast and general engine for large-scale data processing. This change brings Spark to Fedora, allowing easy deployment and development of Spark applications on Fedora. == Detailed Description == Apache Spark is a fast and general engine for large-scale data processing. It supports developing custom analytic processing applications over large data sets or streaming data. Because it has the capability to cache intermediate results in cluster memory and schedule DAGs of computations, Spark programs can run up to 100x faster than equivalent Hadoop MapReduce jobs. Spark applications are easy to develop, parallel, fast, and resilient to failure, and they can operate on data from in-memory collections, local files, a Hadoop- compatible filesystem, or from a variety of streaming sources. Spark also includes libraries for distributed machine learning and graph algorithms. == Scope == * Proposal owners: Currently our Spark package has been accepted into Fedora [1]. It features nearly all of the functionality available from the upstream release. (The missing features -- specifically, Python bindings, the Spark REPL, Kryo-based serialization, primitives for approximate cardinalities of very large sets, and Mesos integration -- were missing from the initial packages due to unavailable dependencies and bundling issues; we're working to close the gap with upstream as quickly as possible.) This work depended upon Fedora 21's improved support for the Scala ecosystem [2]. * Other developers: N/A (not a System Wide Change) * Release engineering: N/A (not a System Wide Change) * Policies and guidelines: N/A (not a System Wide Change) [1] http://pkgs.fedoraproject.org/cgit/spark.git [2] https://fedoraproject.org/wiki/Changes/ImprovedScalaEcosystem _______________________________________________ devel-announce mailing list devel-announce@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel-announce -- devel mailing list devel@xxxxxxxxxxxxxxxxxxxxxxx https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
