• View Page
    • Export to PDF
    • Page history
    • Attachments (3)
    • Page Information
    • View Source
    • Publish to WordPress
    • Convert Gliffy Diagrams
  1. Blog
  2. 2016
  3. May
  4. 19
  5. why spark's mapPartitions transformation is faster than map (calls your function once/partition, not once/element)

Page Information

Title: why spark's mapPartitions transformation is faster than map (calls your function once/partition, not once/element)  
Author: Lester Martin May 19, 2016
Last Changed by: Lester Martin May 20, 2016
Tiny Link: (useful for email) https://martin.atlassian.net/wiki/x/BAD-Aw
Export As: PDF  
Labels
Global Labels (11)
  • spark
  • hadoop
  • apache
  • hdp
  • sandbox
  • tuning
  • tutorial
  • scala
  • software_development
  • hdfs
  • hortonworks
Hot Referrers
(4758) https://www.google.com/
(2047) https://www.google.co.in/
(371) https://www.google.co.uk/
(306) https://www.google.fr/
(279) https://www.google.de/
(276) https://www.google.ca/
(236) https://www.google.co.kr/
(233) https://www.google.com.hk/
(191) https://www.google.es/
(167) https://www.google.com
Recent Changes
Time Editor  
May 20, 2016 01:32 Lester Martin View Changes
May 20, 2016 01:03 Lester Martin View Changes
May 20, 2016 01:00 Lester Martin View Changes
May 20, 2016 00:58 Lester Martin View Changes
May 20, 2016 00:54 Lester Martin  
View Page History
Outgoing Links
External Links (3)
    https://bzhangusc.wordpress.com/2014/06/19/optimize-map-per…
    hortonworks.com/products/sandbox/
    spark.apache.org/docs/latest/programming-guide.html
Manage space
Manage pages
Integrations
{"serverDuration": 213, "requestCorrelationId": "6cef4baaf6083a49"}