Working with Apache Spark - how to pass whole Row to UDF - Spark DataFrame filter


0 votes
asked Mar 17, 2016 by fineFine (1,820 points)
I'm writing filter function for complex JSON dataset with lot's of inner structures.

3 Answers

0 votes
answered Mar 19, 2016 by qb5952 (1,020 points)

This patch passes all tests. It does mean using spark.
commented Mar 20, 2016 by falsonCursor (830 points)
The best way to do this that will be compatible with other
+1 vote
answered Mar 23, 2016 by Gur_576 (420 points)
Credit to http://blog.gmane.org/gmane.comp.lang.scala.spark.user/day=20150703 for the original launch line that I adapted
Open the manifest file and change the line
0 votes
answered Apr 11, 2016 by frc_6228 (250 points)
This problem still exists. Adjust the speed according to your need

Related questions


What is Geekub?

Q&A site for professional and enthusiast programmers, software developers and other technical users. With your help, we hope to work together to build a library of detailed answers to just about any question that is related to programming!







...