Monthly Archives: August 2015

org.apache.spark.SparkException: Job aborted due to stage failure: Task 10095 in stage 1.0 failed 1 times, most recent failure: Lost task 10095.0 in stage 1.0 (TID 10096, localhost): ExecutorLostFailure (executor driver lost)

Spark was throwing a cryptic error while doing a simple Logistic Regression on some text (see exception below). It took me a while to track down the reason.  Basically Spark was running out of memory (would have been nice if … Continue reading

Posted in Uncategorized | Leave a comment

sortByKey at BinaryClassificationMetrics.scala:134 failed : Spark Exception

I had a strange error while getting some metrics (see below). The fix was to add the “key” field to the case class.  In my case it was as following: case class Email( id: String, body: String, headers: Option[Headers] = … Continue reading

Posted in Uncategorized | Leave a comment

Spark: SparkContext: wholeTextFile: InvalidInputException: Input path does not exist

Spark has a handy function `wholeTextFile` for loading files from a diretory into RDD While using it I got a cryptic error (bellow); that took me awhile to debug. The problem was with the file name; for some reason the … Continue reading

Posted in Uncategorized | Leave a comment

Spark MongoDB SecurityException

Problem When trying to run Apache Spark with MongoDB I was getting a SecurityException (below). Solution It seems that “javax.servlet” gets included by multiple packages and not resolved properly.  Hence the fix is to ignore it in one of the … Continue reading

Posted in Uncategorized | Leave a comment